Apparatus and methods for tracking salient features

ABSTRACT

Apparatus and methods for detecting and utilizing saliency in digital images. In one implementation, salient objects may be detected based on analysis of pixel characteristics. Least frequently occurring pixel values may be deemed as salient. Pixel values in an image may be compared to a reference. Color distance may be determined based on a difference between reference color and pixel color. Individual image channels may be scaled when determining saliency in a multi-channel image. Areas of high saliency may be analyzed to determine object position, shape, and/or color. Multiple saliency maps may be additively or multiplicative combined in order to improve detection performance (e.g., reduce number of false positives). Methodologies described herein may enable robust tracking of objects utilizing fewer determination resources. Efficient implementation of the methods described below may allow them to be used for example on board a robot (or autonomous vehicle) or a mobile determining platform.

PRIORITY CLAIM AND CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to co-owned U.S. Provisional PatentApplication Ser. No. 62/053,004 entitled “SYSTEMS AND METHODS FORTRACKING OBJECTS USING SALIENCY”, filed Sep. 19, 2014, which isincorporated herein by reference in its entirety. This application isrelated to co-owned U.S. patent application Ser. No. 13/548,071,entitled “SPIKING NEURON NETWORK SENSORY PROCESSING APPARATUS ANDMETHODS”, filed on Jul. 12, 2012, now U.S. Pat. No. 8,977,582, andco-owned U.S. patent application Ser. No. 13/660,982, entitled “SPIKINGNEURON SENSORY PROCESSING APPARATUS AND METHODS FOR SALIENCY DETECTION”,filed on Oct. 25, 2012, now U.S. Pat. No. 9,218,563, each of theforegoing incorporated herein by reference in its entirety. Thisapplication is also related to co-owned U.S. patent application Ser. No.14/637,138 entitled “SALIENT FEATURES TRACKING APPARATUS AND METHODSUSING VISUAL INITIALIZATION”, and Ser. No. 14/637,191 entitled“APPARATUS AND METHODS FOR SALIENCY DETECTION BASED ON COLOR OCCURRENCEANALYSIS” issued as U.S. Pat. No. 9,870,617 on Jan. 16, 2018, each filedcontemporaneously herewith on Mar. 3, 2015, each also incorporatedherein by reference in its entirety.

COPYRIGHT

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever.

BACKGROUND

Field of the Disclosure

The present disclosure relates to, inter alia, computerized apparatusand methods for detecting features in sensory information.

Description of Related Art

Saliency is a term often used in psychophysics and neuroscience, whereit may be defined as the ability to attract attention to a particularstimulus. In the visual modality, saliency may be attributed to objectsor visual patterns. In primates, there are at least five general aspectsof the visual scene which spontaneously attract attention (a phenomenonoften called pop-out):

-   -   1. Color—a unique color surrounded by another color or colors        (e.g., a red spot in sea of green) will involuntarily trigger        attention.    -   2. Orientation—a bar of one orientation surrounded by bars of        another orientation    -   3. Contrast (luminance)—an area characterized by large contrast        (a bright thing mixed with very dark things)    -   4. Motion—an object moving in a different direction or speed        that surroundings.    -   5. Faces—only for humans and some primates, image resembling a        face

The biological origin of this behavior is debated. One of the plausiblemechanisms is local inhibitory interaction in visual cortex (and perhapsthalamus). Such inhibitory connections arise between neurons whichrepresent things often co-occurring. For example, a segment of verticalline (observed by a spatially restricted receptive field of a neuron) istypically a fragment of a larger line. A color blob seen in a receptivefield of a neuron is most often a fragment of a larger blob of the samecolor. Appearance of a feature (e.g., orientation) which is not expectedin a given context (e.g., vertical line segment surrounded by a largenumber of horizontal lines) is statistically much more rare, thereforeinhibitory connections for such occurrence are much weaker. Inconsequence, the response of a neuron representing such “unexpectedfeatures” is stronger, as the otherwise present inhibition is lacking.

From an evolutionary point of view, sensitivity to such unexpectedstimuli is beneficial. It allows one to quickly detect anomalies in thevisual scene, which often indicate behaviorally relevant information(e.g., a ripe, red fruit among green leaves, a dangerous predator hidingin the bushes).

SUMMARY

One aspect of the disclosure relates to a non-transitorycomputer-readable storage medium. In one implementation, the medium hasinstructions embodied thereon, the instructions being executable by aprocessing apparatus to perform a method of tracking an object in asequence of images. In one variant, each of the images is comprised ofpixels, and the method includes: identifying a salient area in an imageof the sequence of images; and initializing an object tracking processbased at least on the identifying the salient area.

In one variant, the initializing is configured to reduce a number ofoccurrences of false positives associated with detecting the object in asubsequent image of the plurality of images, as compared to detectingthe object in the subsequent image in absence of the initializing.Individual pixels of the image of the sequence of images arecharacterized by e.g., one or more pixel characteristics (e.g.,chromacity parameter); and the identifying the salient area comprisesdetermining a saliency distribution of pixels of the image of thesequence of images based on at least one of the one or more pixelcharacteristics.

In a further variant, the determination of the saliency distributioncomprises determining a spatial distribution of pixel color distance inthe image, the pixel color distance being determined based at least on adistance measure between the chromaticity parameter of individual pixelsof the image and a reference chromaticity.

In yet another variant of the medium, identifying the salient area inthe image comprises: partitioning the image into a plurality ofpartitions, individual ones of the plurality of partitions comprising aplurality of adjacent pixels in a contiguous area; convolving individualones of the plurality of partitions with one or more spatial filtermasks to produce a plurality of convolution magnitudes, individual onesof the one or more spatial filters characterized by filter orientation;and determining the salient area corresponding to a least frequentlyoccurring filter convolution magnitude above a threshold.

In another variant of the medium, the salient area is identified basedat least on analysis of displacement of pixels of the image relative topixels of a preceding image of the sequence of images. The analysis ofdisplacement comprises e.g., determination of a motion map of pixels ofthe image, the motion map at least characterizing magnitude anddirection of motion of pixels in the image associated withrepresentation of the object; and the saliency map determination isconfigured based at least on an additional information related to motionof the object.

In yet another variant of the medium, the image of the sequence ofimages is obtained by a sensor configured to sample energy reflectedfrom an environment; individual pixels of the image are characterized bya range channel corresponding to a range value between the sensor and alocation within the environment; and the salient area is determinedbased at least on information related to a given range to the object anda comparison of range channel value associated with pixels in thesalient area to the range to the object.

In some implementations, the method implemented by the medium maycomprise determining a shape characteristic of the feature. Theparameter may comprise the characteristic.

In some implementations, the characteristic may be configured tocharacterize a circularity of the feature.

Another aspect of the disclosure relates to a method of operating arobotic device.

Yet another aspect of the disclosure relates to a method of tracking anobject in a sequence of images. The method may comprise: determining asaliency measure based on analysis of pixels of an image of the sequenceof imaged; and providing the saliency measure to a tracking process, thetracking process configured to determine representation of the object inindividual ones of the sequence of images.

In some implementations, the identity of the tracked salient object maybe estimated via computation of color distance, such as between thepixels representing the salient object and a previously initializedcolor prototype.

In some implementations, the determination of the saliency measure maycomprise determining orientation of the feature to initialize a colordistance computation to maximize discriminability between the target andthe background.

In some implementations, the method may comprise determining orientationsaliency to initialize a tracker pick the salient object.

In some implementations, the method may comprise using distance saliencyto initialize a tracker.

In some implementations, the method may comprise using color saliency topick a number of object colors to maximize discriminability from thebackground.

Still another aspect of the disclosure relates to a controller of arobotic apparatus. The controller may comprise a sensor processingcomponent configured to analyze sensor input. The sensor input analysismay be configured to: determine visual (e.g., color) saliency;initialize a tracking process using the color saliency; determine asalient feature; operate the tracking process to detect the feature; andcommunicate tracking information to a controller component configured toadjust trajectory of the robotic device in accordance with acharacteristic of the feature.

In some implementations, the salient feature determination may beconfigured based on color saliency. The salient feature information maybe automatically provided to the tracking process. The feature maycomprise an object. The trajectory adjustment may be configured toreduce spatial separation between the object and the apparatus.

In some implementations, the controller may comprise a manipulator. Thetrajectory adjustment may comprise operating the manipulator to pick upthe object.

Yet another aspect of the present disclosure relates to a roboticapparatus. In one embodiment, the robotic apparatus may be configured totrack an object and include: a camera; and a non-transitorycomputer-readable apparatus including a storage medium havinginstructions contained therein, the instructions being configured tocause the robotic apparatus to, when executed: obtain a plurality ofimages via the camera while navigating a trajectory associated with aphysical task; identify a salient area in a given image of the pluralityof images, the identification of the salient area being based at leaston a motion component associated with the object and a motion componentassociated with the robotic apparatus; initialize an object trackingprocess based at least on the identification of the salient area in thegiven image, the object tracking process including a detection of theobject from the salient area; and adjust the trajectory associated withthe physical task based on the detection of the object from the salientarea.

These and other objects, features, and characteristics of the systemand/or method disclosed herein, as well as the methods of operation andfunctions of the related elements of structure and the combination ofparts and economies of manufacture, will become more apparent uponconsideration of the following description and the appended claims withreference to the accompanying drawings, all of which form a part of thisspecification, wherein like reference numerals designate correspondingparts in the various figures. It is to be expressly understood, however,that the drawings are for the purpose of illustration and descriptiononly and are not intended as a definition of the limits of thedisclosure. As used in the specification and in the claims, the singularform of “a”, “an”, and “the” include plural referents unless the contextclearly dictates otherwise.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a graphical illustration depicting a robotic apparatuscomprising an feature detection apparatus of the disclosure, configuredfor autonomous navigation, in accordance with one or moreimplementations.

FIG. 2A illustrates use of pixel likelihood for feature detection in aYUV color space image, according to one or more implementations.

FIG. 2B illustrates saliency map comprising multiple salient areasdetermined based on pixel likelihood determination, according to one ormore implementations.

FIG. 3 illustrates use of color distance for determining a salientfeature in an image frame shown in FIG. 2A, according to one or moreimplementations.

FIG. 4 illustrates saliency maps obtained using a plurality of imagecharacteristics, according to one or more implementations.

FIG. 5 illustrates use of a saliency map configured based on colorlikelihood as a priming map for detecting salient feature, according toone or more implementations.

FIG. 6 illustrates use of multiple saliency maps for detecting a targetobject according to one or more implementations.

FIG. 7 is a functional block diagram depicting a system comprisingsalient feature detection apparatus, according to one or moreimplementations.

FIG. 8 is a functional block diagram depicting the salient featuredetection apparatus of, e.g., FIG. 7, according to one or moreimplementations.

FIG. 9A-9D illustrates gestures of a human operator used forcommunicating control indications to a robotic device, gestures beingdetectable by a salient feature detection system of, e.g., FIGS. 6-7, inaccordance with one or more implementations.

FIG. 10 illustrates use of saliency and/or object detection methodologyby an unmanned robotic apparatus configured for autonomous navigation,in accordance with one or more implementations.

FIG. 11 is a logical flow diagram illustrating a method of dataprocessing useful for determining features in an image, in accordancewith one or more implementations.

FIG. 12 is a logical flow diagram illustrating a method of determining asaliency map based on likelihood information, in accordance with one ormore implementations.

FIG. 13 is a logical flow diagram illustrating a method of trajectorynavigation by a robotic device using salient feature detection, inaccordance with one or more implementations.

All Figures disclosed herein are ©Copyright 2015 Brain Corporation. Allrights reserved.

DETAILED DESCRIPTION

Implementations of the present disclosure will now be described indetail with reference to the drawings, which are provided asillustrative examples so as to enable those skilled in the art topractice the present technology. Notably, the figures and examples beloware not meant to limit the scope of the present disclosure to a singleimplementation, but other implementations are possible by way ofinterchange of or combination with some or all of the described orillustrated elements. Wherever convenient, the same reference numberswill be used throughout the drawings to refer to same or like parts.

Although the system(s) and/or method(s) of this disclosure have beendescribed in detail for the purpose of illustration based on what iscurrently considered to be the most practical and preferredimplementations, it is to be understood that such detail is solely forthat purpose and that the disclosure is not limited to the disclosedimplementations, but, on the contrary, is intended to covermodifications and equivalent arrangements that are within the spirit andscope of the appended claims. For example, it is to be understood thatthe present disclosure contemplates that, to the extent possible, one ormore features of any implementation may be combined with one or morefeatures of any other implementation

In the present disclosure, an implementation showing a singularcomponent should not be considered limiting; rather, the disclosure isintended to encompass other implementations including a plurality of thesame component, and vice-versa, unless explicitly stated otherwiseherein.

Further, the present disclosure encompasses present and future knownequivalents to the components referred to herein by way of illustration.

As used herein, the term “bus” is meant generally to denote all types ofinterconnection or communication architecture that is used to access thesynaptic and neuron memory. The “bus” could be optical, wireless,infrared or another type of communication medium. The exact topology ofthe bus could be for example standard “bus”, hierarchical bus,network-on-chip, address-event-representation (AER) connection, or othertype of communication topology used for accessing, e.g., differentmemories in pulse-based system.

As used herein, the terms “computer”, “computing device”, and“computerized device”, include, but are not limited to, personalcomputers (PCs) and minicomputers, whether desktop, laptop, orotherwise, mainframe computers, workstations, servers, personal digitalassistants (PDAs), handheld computers, embedded computers, programmablelogic device, personal communicators, tablet or “phablet” computers,portable navigation aids. J2ME equipped devices, smart TVs, cellulartelephones, smart phones, personal integrated communication orentertainment devices, or literally any other device capable ofexecuting a set of instructions and processing an incoming data signal.

As used herein, the term “computer program” or “software” is meant toinclude any sequence or human or machine cognizable steps which performa function. Such program may be rendered in virtually any programminglanguage or environment including, for example, C/C++, C#, Fortran,COBOL, MATLAB™, PASCAL, Python, assembly language, markup languages(e.g., HTML, SGML, XML, VoXML), and the like, as well as object-orientedenvironments such as the Common Object Request Broker Architecture(CORBA), Java™ (including J2ME, Java Beans), Binary Runtime Environment(e.g., BREW), and other languages.

As used herein, the teems “connection”, “link”, “synaptic channel”,“transmission channel”, “delay line”, are meant generally to denote acausal link between any two or more entities (whether physical orlogical/virtual), which enables information exchange between theentities.

As used herein the term feature may refer to a representation of anobject edge, determined by change in color, luminance, brightness,transparency, texture, and/or curvature. The object features maycomprise, inter alia, individual edges, intersections of edges (such ascorners), orifices, and/or curvature

As used herein, the term “memory” includes any type of integratedcircuit or other storage device adapted for storing digital dataincluding, without limitation, ROM. PROM, EEPROM, DRAM, Mobile DRAM,SDRAM, DDR/2 SDRAM, EDO/FPMS, RLDRAM, SRAM, “flash” memory (e.g.,NAND/NOR), memristor memory, and PSRAM.

As used herein, the terms “processor”, “microprocessor” and “digitalprocessor” are meant generally to include all types of digitalprocessing devices including, without limitation, digital signalprocessors (DSPs), reduced instruction set computers (RISC),general-purpose (CISC) processors, microprocessors, gate arrays (e.g.,field programmable gate arrays (FPGAs)), PLDs, reconfigurable computerfabrics (RCFs), array processors, secure microprocessors, andapplication-specific integrated circuits (ASICs). Such digitalprocessors may be contained on a single unitary IC die, or distributedacross multiple components.

As used herein, the term “network interface” refers to any signal, data,or software interface with a component, network or process including,without limitation, those of the FireWire (e.g., FW400, FW800, and/orother FireWire implementation.), USB (e.g., USB2), Ethernet (e.g.,10/100, 10/100/1000 (Gigabit Ethernet), 10-Gig-E, etc.), MoCA, Coaxsys(e.g., TVnet™), radio frequency tuner (e.g., in-band or OOB, cablemodem, etc.), Wi-Fi (802.11), WiMAX (802.16), PAN (e.g., 802.15),cellular (e.g., 3G, LTE/LTE-A/TD-LTE, GSM, and/or other cellularinterface implementation) or IrDA families.

As used herein, the terms “pulse”, “spike”, “burst of spikes”, and“pulse train” are meant generally to refer to, without limitation, anytype of a pulsed signal, e.g., a rapid change in some characteristic ofa signal, e.g., amplitude, intensity, phase or frequency, from abaseline value to a higher or lower value, followed by a rapid return tothe baseline value and may refer to any of a single spike, a burst ofspikes, an electronic pulse, a pulse in voltage, a pulse in electricalcurrent, a software representation of a pulse and/or burst of pulses, asoftware message representing a discrete pulsed event, and any otherpulse or pulse type associated with a discrete information transmissionsystem or mechanism.

As used herein, the term “receptive field” is used to describe sets ofweighted inputs from filtered input elements, where the weights may beadjusted.

As used herein, the term “Wi-Fi” refers to, without limitation, any ofthe variants of IEEE-Std. 802.11 or related standards including 802.11a/b/g/n/s/v and 802.11-2012.

As used herein, the term “wireless” means any wireless signal, data,communication, or other interface including without limitation Wi-Fi,Bluetooth, 3G (3GPP/3GPP2), HSDPA/HSUPA, TDMA, CDMA (e.g., IS-95A,WCDMA, and/or other wireless interface implementation.), FHSS, DSSS,GSM, PAN/802.15, WiMAX (802.16), 802.20, narrowband/FDMA, OFDM, PCS/DCS,LTE/LTE-A/TD-LTE, analog cellular, CDPD, RFID or NFC (e.g., EPC GlobalGen. 2, ISO 14443, ISO 18000-3), satellite systems, millimeter wave ormicrowave systems, acoustic, and infrared (e.g., IrDA).

The present disclosure provides, among other things, apparatus andmethods for determining salient features in sensor data. In someimplementations, the sensor data may comprise images of a visual sceneobtained using a video camera. The video information may comprise forexample multiple streams of frames received from a plurality of camerasdisposed separate from one another. Individual cameras may comprise animage sensor (e.g., charge-coupled device (CCD), CMOS device, and/or anactive-pixel sensor (APS), photodiode arrays, and/or other sensors). Inone or more implementations, the stream of frames may comprise a pixelstream downloaded from a file. An example of such a file may include astream of matrices of red green blue RGB values (e.g., refreshed at a 25Hz or other suitable frame rate). It will be appreciated by thoseskilled in the art when given this disclosure that the above-referencedimage parameters are merely exemplary, and many other imagerepresentations (e.g., bitmap, luminance-chrominance (YUV, YCbCr),cyan-magenta-yellow and key (CMYK), grayscale, and/or other imagerepresentations) may be applicable to and useful with variousimplementations of the present disclosure. Furthermore, data framescorresponding to other (non-visual) signal modalities such as sonograms,infrared (IR), radar or tomography images may be compatible with theprocessing methodology of the disclosure, or yet other configurations.

The salient feature detections processing methodology described hereinmay enable a controller of a robotic device to detect targets and/orobstacles that may be present during trajectory navigation by thedevice, e.g., 160 of FIG. 1.

FIG. 1 depicts a mobile robotic apparatus comprising a feature detectionapparatus configured, e.g., in accordance with the exemplaryimplementations illustrated below. The robotic apparatus 160 maycomprise a camera 166. The camera 166 may be characterized by a field ofview 168 (e.g., an extent of the observable world that may be capturedby the camera lens at a given moment). The camera 166 may provideinformation associated with objects within the field of view 168. Insome implementations, the camera 166 may provide frames of pixels ofluminance and/or color, refreshed at 25 Hz frame rate. However, it willbe appreciated that, in some implementations, other frame rates may beused (whether constant or variable), as may other types of informationprovided by the camera(s) 166.

One or more objects (e.g., a floor 170, a stationary object 176, amoving object (e.g., ball 174), and/or other objects) may be present inthe camera field of view. The motion of the objects may result in adisplacement of pixels representing the objects within successiveframes, such as is described in co-owned U.S. patent application Ser.No. 13/689,717 filed on Nov. 29, 2012 and entitled “APPARATUS ANDMETHODS FOR OBJECT DETECTION VIA OPTICAL FLOW CANCELLATION”, issued asU.S. Pat. No. 9,193,075 on Nov. 24, 2015, which is incorporated hereinby reference in its entirety.

When the robotic apparatus 160 is in motion, such as shown by arrow 164in FIG. 1, motion of the objects within the camera 166 field of view 168(e.g., denoted by arrows 172, 178, 180 in FIG. 1) may comprise theself-motion component and the object motion component. By way of anon-limiting example, motion of objects in FIG. 1 may comprise apparentmotion 180 of the stationary background 176 and the boundary (e.g., thecomponent 172 associated with the floor boundary); (ii) component 178associated with the moving ball 174 that comprises a superposition ofthe ball displacement and motion of the camera; and/or other components.As noted previously, determination of the ball 174 motion may beparticularly challenging when the camera 160 is in motion (e.g., duringpanning) and/or when the field of view is changing (e.g., when zoomingin/out).

Using relative aspects of an image may allow mitigation of detectionproblems that may arise due to lighting variance. For example, theabsolute color of the object and/or background may vary widely dependingon illumination, but relative saliency may be preserved. These aspectsof saliency may make it a useful perceptual input to an autonomousrobotic platform.

In some, implementations, the saliency of an item (such as an object, aperson, a pixel, and/or other) may be described by a characteristic bywhich the item may stand out relative to its neighbors. Saliency mayarise from, for instance, contrast between the item and itssurroundings, such as a black object on a white background, or a roughscrape on a smooth surface. For example, a salient vehicle may comprisea vehicle that may be moving differently (e.g., going slower/faster thanthe rest of the traffic, weaving from lane to lane) compared to the restof the traffic. A salient object for target approach may comprise astationary and/or moving ball on a moving background due to self-motionby the vehicle.

Saliency may be determined in a number of ways by analyzing input imagesrepresented in one of the typical ways (e.g., grayscale, RGB, YUV, HSV,and/or other color models). In some implementations, an input may becharacterized by a spatial extent (e.g., an image or array of pixels)and by color and/or luminance. Saliency may be determined based on alikelihood characterizing the input. By way of an illustration, for animage I comprising a plurality of pixels i(x,y), saliency at a pixellocation (x,y) may be determined based on a likelihood of occurrence ofthe pixel value i in the image I. Lower likelihood may correspondgreater the saliency at the location (x,y), in accordance with someimplementations.

The saliency may be determined using a back projection of the imagepixel likelihood information. In some implementations, the saliency ofindividual pixels of the image may be characterized by the number ofpixels representing the same exact value in the rest of the image. Forexample an 8-bit image may comprise 300000 pixels having value of 128and 1000 pixels carrying value of 0. For such image, less frequentlyoccurring pixels (e.g., with the value of 0) may be referred to as moresalient.

In some implementations, saliency may be determined using a modelstatistics approach. For instance, a model of expected pixeldistribution may be utilized. Parameters of a given pixel distribution(e.g., Gaussian, Poisson, uniform), may be determined. In someimplementations (e.g., of Gaussian distribution) the color distributionmodel may be characterized by two parameters, e.g., mean and a standarddeviation. Pixel with values outside a given range (e.g., outside range[m−N1×stdev, m+N2×stdev] may be regarded as salient, where m denotesmean pixel value, stdev denotes standard deviation, N1, N2 denotemultiplicative parameters (e.g. selected between value of 2 and value of6). In some implementations (e.g., of Poisson distribution) the colordistribution model may be characterized by one parameter, e.g., expectedvalue/variance. Model of pixel color distribution may be determinedbased on analysis of one or more prior images and/or using an expectedcolor distribution for a given scene (e.g., images of the sky, images ofgrass, and/or other environment).

In some implementations, the saliency determination may be realized byone or more of the operations described with respect to FIGS. 2A-2B,and/or FIGS. 11-12. Pixel likelihood may be characterized by pixeloccurrence frequency, e.g., histogram of pixel values for an image. Fora given image I, e.g., 200 in FIG. 2A, histogram H of pixel values maybe determined. In some implementations of a single channel images (e.g.,grayscale) the histogram may be determined using single channel values(e.g., intensity). In one or more implementations a multi-channel images(e.g., RGB, YUV, HSV, and/or other representations) the histogram may bedetermined for individual channels and or multiple channels.

Using the image histogram H, a back-projection matrix B may bedetermined. Dimensions of the matrix B may comprise dimensions of theoriginal image I. In some implementations, the back-projection B may bedetermined as follows: individual pixel b(x,y) of the back-projectedimage may be assigned a value equal to the value (count) in thehistogram for the pixel value i(x,y) of the original image I at thelocation (x,y):B[x,y]=H(I[x,y])  (Eqn. 1)

By way of an illustration, if a pixel at location (1,1) in the originalimage is of value i(1,1)=0, and there are 1000 pixels with values of 0in the image, the corresponding values b(1,1)=1000 in the back-projectedimage B. For a binned histogram implementation, a pixel with a valuewithin a bin range may be assigned the bin count value.

A saliency metric S may be determined from the back-projection B. Insome implementations, the saliency metric may comprise a matrixdetermined as follows:S[x,y]=1/(B[x,y]+C)  (Eqn. 2)where C denotes a parameter configured constant or dynamicallyadjustable. In one or more implementations, the saliency metric may bedetermined as:S[x,y]=C−B[x,y]/f(B)  (Eqn. 3)where f(B) may denote a norm operation. Other implementations may beutilized for a given level of target range and/or precision of saliencyvalues.

In some implementations, determining a full histogram may beimpractical, e.g., when pixels may comprise values in a very largespace. For example, in 24 bit RGB (Red, Green, Blue) image may comprise16.7 million possible pixel values. For most images every value of theimage may correspond to a few (e.g., less than 1000 in someimplementations) pixels. Determining the histogram for saliency in suchcase may be computationally taxing and/or may provide a weak signal, asthe majority of pixels may be considered as salient (unique). To improvethe dynamic range of such maps, binning may be used. That is, individualbins of the histogram do not correspond to a single value, but rather arange of values. The number of bins may be selected from the rangebetween 2 and 10 per channel. For a 3-channel image (e.g., RGB, HSV orYUV) the total number of bins per image may be between 8 and 1024. Agiven bin may correspond to a range of values (e.g., [0 to 63] for bin 1of 4-bins for 8-bit channel). Pixels with values falling within the binrange (or a bin with multiple ranges for a multi-channel histogram) maycause increment of the bin value (e.g., count).

Color spaces are often three dimensional. In these cases, the bins maycorrespond to cubes tiling the color space. In some implementations, thecolor space may be tiled uniformly in each direction (e.g., RGB colorcube may be subdivided into 27 sub-cubes with individual sub-cubespanning ⅓ of individual dimension). In other color spaces (e.g., YCbCr,YUV, HSV, and/or other color spaces) where luminance may be representedseparately from chrominance (color information), non-uniform binning maybe employed. By way of an illustration, larger bins may be used to tilethe luminance channel dimension; smaller bins may be used forchrominance channel dimensions. In some implementations, clipping therange for bins and/or dynamically placed bin edges may be used in orderto enable control of the dynamic range of the saliency map forindividual aspects (e.g., color channels) of the image. In someimplementations, a single bin may be used for luminance channel with aplurality of bins for the chrominance channels. Saliency informationderived from such implementations may be configured invariant toluminance changes.

FIG. 2A illustrates use of pixel likelihood information for detecting afeature in an image, according to one or more implementations. The image200 in FIG. 2A may comprise an image obtained with a camera of a roboticdevice (e.g., 166 of 160 in FIG. 1) during navigation of a trajectory.In some implementations, the image 200 may comprise a previously recodedimage that may be accessed from a storage medium (e.g., flash, optical,magnetic storage). Image 200 shown in FIG. 2A may comprise luminositychannel Y and two chromaticity channels U,V. The image 200 may comprise320×240 pixels for Y channel, 160×120 pixels for each of U,Vchromaticity channels. Pixel values for individual (YUV) channels may becharacterized by 8-bit resolution thereby corresponding to pixel valuerange between 0 and 255. In some implementations, the image 200 maycomprise an RGB color space image comprising 320×240 pixels forindividual red (R), green (G), and blue (B).

The image 200 may be characterized by a 3-dimensional color space (e.g.,cube 206). The color space 206 may be partitioned into one or more bins,e.g., 256 bins in the implementation illustrated in FIG. 2A. Luminositychannel may be partitioned into 4 bins, U,V chromaticity channels may bepartitioned into 8 bins each. Histogram of the image 200 may bedetermined as shown by arrow 202 in FIG. 2A. Pixels of the image 200 maybe analyzed and assigned to one of the bins of the binned color space,thereby producing pixel likelihood distribution (the histogram). A backprojection 210 of the likelihood information onto the image space may bedetermined (as shown by arrow 204 in FIG. 2A). In some implementations,the back projection determination may comprise, for a given location(x,y) in the image 210, assigning a value corresponding to value thehistogram bin that corresponds to the pixel at that location (x,y) inthe original image 200. In some implementations, the back projection 210may be scaled to a given e.g., 8-bit range, using the followingtransformation:B1(x,y)=256−B(x,y)/C

As may be seen in FIG. 2A, majority of the pixels in the back-projectedimage 210 are assigned a value corresponding to black color. A portionof pixels are assigned a value corresponding to gray color; a smallerportion of pixels (e.g., denoted by arrow 212 in FIG. 2A) are assignedvalue corresponding to white. The white area in the image 210 in FIG. 2Amay be referred to as salient. Relating image 210 to the original image200, white pixels in image 210 correspond to color of red hues (V>0 inYUV color model) in the image 200. Black pixels in the image 210 maycorrespond to green colors (V<1, U<0.5 in some implementations of YUVcolor model) in the image 200. In some implementations, salient area(e.g., 212 in image 210) may be determined based on occurrence of aplurality pixels with sufficient saliency (e.g., saliency greater than agiven threshold of, e.g., 95%) proximate to one another. A plurality ofproximate pixels may be amalgamated to produce a salient area. Number ofthe proximate pixels (salient pixel neighborhood) may be selected from arange that may correspond to 0.002% and 2% of a given image dimension.By way of an illustration, for an (x,y) image with x-dimension of 1000pixels and y dimension of 2000 pixels, salient pixel neighborhood maycorrespond to between 2 pixels and 20 pixels in x-dimension and between4 pixels and 40 pixels in y-dimension. Pixel saliency may be determinedbased on pixel likelihood be in a target range. In some implementations,the target range may be selected to correspond to between 5% and 0.01%least likely pixels in the image.

In some implementations, saliency map produced based on analysis ofpixel value likelihood (e.g., using Eqn. 1) may comprise two or moreareas of high saliency. FIG. 2 illustrates saliency map comprisingmultiple salient areas determined based on pixel likelihooddetermination, according to one or more implementations. Image 220 inFIG. 2B may be obtained by a robotic device (e.g., 160 in FIG. 1) duringtask execution. In some implementations, the image 220 may comprise320×240 pixel matrix for luminosity Y channel, 160×120 pixel matricesfor individual ones of U,V chromaticity channels. Pixel values forindividual (YUV) channels of image 220 may be characterized by 8-bitresolution thereby corresponding to pixel value range between 0 and 255.Image 220 may comprise a target object (e.g., 222 in FIG. 2B).

Saliency map image 230 may be obtained using back-projection of theimage 220 histogram determined using, e.g., 256 bins (4,8,8),respectively, of YUV color space. Areas corresponding to low pixel valueoccurrence likelihood (e.g., bin count in the histogram below 10%) maybe identified as salient (e.g., areas 232, 234, 236 in FIG. 2B). Areacorresponding to the target object of interest (e.g., the ball 222) maybe determined. The target object area may be determined based onanalysis of color distance, e.g., as described below with respect toFIG. 3, analysis of motion, circularity analysis (e.g., as describedbelow with respect to FIG. 4), and/or other approaches.

In some implementations, saliency information may be determined based onanalysis of difference between a value of a pixel and values of pixelsin a surrounding region. In some implementations, an image I may beconvolved with a low-pass kernel (e.g., a Gaussian kernel) to create alow-passed image F. In some implementations, wherein the image I maycomprise 320 pixels by 200 pixel matrix with pixel density of Z pixelsper mm, the low pass filter half power (e.g., 3 dB) dimensions may beselected between 7 and 50 pixels. It will be recognized by those skilledin the arts that various filter implementations (e.g., exponential,Chebyshev, Hamming) may be utilized with the saliency determinationapproaches described herein. In some implementations, filter x-dimensionmay be selected different from the y-dimension. Filter dimensions may beadjusted in accordance with operational task characteristics, e.g.,image size, resolution, pixel density, potential target size, amount ofnoise in the image. By way of an illustration, a lower resolution/lowerpixel count image may correspond to a smaller filter size when expressedin terms of pixels; higher resolution/greater pixel count image maycorrespond to a larger filter size when expressed in terms of pixels.

Saliency map S may be determined by assigning to individual pixels adistance measure between value of the pixel in the original image I andvalue of the pixel (at the same location) in the low-passed image F. Insome implementations, the saliency map may be expressed as follows:S[x,y]=d(F[x,y],I[x,y]),  (Eqn. 4)where F denotes the low passed version of the image I, and d( ) denotesa distance metric.

In some implementations, a pixel characterized by a greater distance(e.g., pixel having a value that is further than an average value of itssurround) may be referred to as (more) salient, compared to pixels whosevalues may be closer to the average surround value. The distance metricd( ) may comprise a Euclidean distance (e.g., determined on the valuesof pixels), some form of rectilinear distance (e.g., sum of absolutevalues of differences between respective dimensions, e.g. Y, U, V valuesof the pixel). In some implementations, distance determination may beconfigured to emphasize or suppress one aspect (e.g., luminance) vsanother aspect (e.g., chrominance). By way of an illustration, distancedetermination may be configured to utilize chrominance without usingluminance.

In some implementations, saliency may be determined based on otheraspects of the image, e.g., dominant orientation of features across animage. The orientation may be characterized by a set of parametersincluding one or more of angle, phase, frequency, contrast, and/or otherparameters. An orientation map (also referred as feature orientationdistribution) may be generated from the original input image whose pixelvalues may convey information related to orientation property (e.g.,angle) of the original image. In some implementations, the orientationmap may be determined based on convolving the initial image (e.g., 200in FIG. 2) with a plurality of directional filters. Individualdirectional filters may comprise a spatial mask characterized by apreferred direction (e.g., 0°, 45°, 90° and/or other direction). Themask size may be selected between 3×3 pixels and 31 by 31 pixels whenprocessing the image 200 comprising 320×200 pixels. Larger masks may beused when processing larger image sizes. In one or more implementations,the orientation map may be determined using local spatial frequencytransformation (e.g. spatial Fourier transform over an area of animage). The orientation map may comprise values indicating most activefilter (e.g., filter index with greatest convolution magnitude). Theorientation distribution may be analyzed to produce saliency informationby, e.g., using orientation likelihood, pixel distance and/or otherapproaches. In some implementations, orientation distribution may beconfigured based on an index of activated spatial filter (e.g., filterwith predominantly vertical sensitivity may be assigned index value of1, filter with predominantly horizontal sensitivity may be assignedindex value of 3, and/or other index implementation). Analysis of theorientation distribution may comprise for example determining leastfrequently activated filter, e.g., by determining least frequentlyoccurring index. In some implementations, the analysis may comprisesorting of the orientation map entries. Saliency information determinedfrom the orientation map may reflect anomalies based on orientationaspects of group(s) of pixels the original image.

The saliency detection methodology described herein may be applied to asequence of images. Saliency may be determined based on a difference(also referred to motion saliency) between consecutive images in thesequence. In some implementations, optical flow. e.g., a vector fieldrepresenting direction and/or velocity of motion of pixels betweensuccessive frames, may be used for determining motion saliency. Avariety of approaches may be used for estimating motion from a sequenceof images. Motion information may be presented as a two dimensionalmotion vector map. The motion vector map may be regarded as an image;individual vector components (e.g., x motion y motion) may be regardedas image channels. Motion information may be processed in order todetermine saliency using, e.g., methodology described herein. In someimplementations, global/local statistical analysis along individualand/or a combination of individual motion channels (e.g., x-motion,y-motion, motion magnitude, motion direction, and/or other parameters)may be used for saliency determination. Analyses of motion informationmay reveal anomalies and/or indicate motion saliency.

In some implementations, wherein images may be obtained from a movingplatform, majority of the motion may comprise motion along a particulardirection (e.g., corresponding to the direction of motion of thecamera). Such motion may be referred to as the background motion. Anindependently moving object may be perceived as salient. In someimplementations, a robotic device may control its movements and/orobserve the environment through a camera. Certain motor commands (e.g.,forward, left-turn) may be associated with particular motion flow fields(e.g., left-turn command may generate a relatively uniform motion flowtowards right). Observed features of the motion field may be used forsaliency detection. By way of an illustration, for a given motor command(e.g., left turn), deviation from an expected motion field (e.g.,predominant motion to the right) may comprise salient features. Forexample, if a robotic agent is moving forward, a motion fieldorigination from the direction of motion and radially expanding towardsthe sides (looming) may be expected. A convergent portion of the motionflow field may indicate an object moving away from the robotic agent.Such object may be marked as salient in some implementations (e.g., whenthe agent is supposed to follow a moving target).

In some implementations (e.g., comprising distance to object measurementusing LiDAR, ultrasonic range finder, stereo imager, structured lighttime of flight measurement, and/or other approaches), saliencyinformation may be determined based on analysis of distance to objects(also referred to as depth. A object disposed at a distance that maydiffer from the rest of the scene may be referred as salient.Depth-derived saliency map may be utilized for navigation of autonomousflying drones for detection of obstacles.

Saliency may be used to identify the presence of objects in visualscene. In some implementations, additional information may be utilizedfor identifying object(s) of interest. Combining the saliency withobject identity masks may allow for robust tracking, even in relativelycomplex conditions for object recognition. In some implementations,feature tracking and/or object tracking may comprise priming thetracking process by exposing an object of interest (e.g., ball) to thesensor (e.g., digital camera) configured to provide information to thetracking process. The priming may correspond to one or more digitalimages (priming images) that may be available to the tracking process.Saliency determination processing may be applied to the priming imagesin order to determine a salient feature within the one or more primingimages. The tracking process may be configured to track the salientfeature through one or more images subsequent to the priming image(s),the tracking configured based on an occurrence of a similarly salientfeature in an image subsequent to the priming image. The saliencysimilarity may be configured based on a parameter used for determiningthe salient feature in the priming image(s). By way of an illustration,when pixel color may be used for determining salient feature in thepriming image, subsequent images may be analyzed to determine saliencybased on the pixel color; when color distance may be used fordetermining salient feature in the priming image, subsequent images maybe analyzed to determine saliency based on the color distance color.Other parameters (e.g., orientation, circularity, shape, motion, colorsurface fill, physical distance, motion flow, locality, kinematic mask),a combination thereof and/or other parameters may be utilized forsaliency determination, as described herein. In some implementations,tracking process may be provided with information related to motion ofthe sensor (e.g., the camera 166 of vehicle 160 in FIG. 1). Motioninformation may be utilized in order to adjust salient area location inaccordance with camera motion (e.g., when panning, turning, and/or othermaneuvers).

In some implementations, objects may be characterized by color. Coloralone, however, may not be sufficient for feature tracking due tochanges in absolute value of the object color due to, e.g., illuminationchanges, object orientation, and/or other causes. To illustrate, a redball on an orange carpet may become brown when in shadow. In fact, theorange carpet may become red in the shadow. Relying only on absolutecolor value in this case may lead to substantial errors in objectdetection and/or tracking.

In some implementations, color information may be sufficient todistinguish between the object of interest and other salient objects (ifpresent). By way of an illustration, the red ball may likely remain themost salient object whether in shadow or not. Among the salient objectsdetected on the floor, it may likely still be the reddest. A trackingmethod comprising color saliency evaluation may lead to much bettertracking performance, compared to methods that to not employ saliency.

In some implementations, color distance may be used for determiningsilent objects. Color distance may be determined in a number of ways,depending on the image representation. In some implementations,Euclidean distance in the color space may be used. In one or moreimplementations, a rectilinear distance in a multi-dimensional colorspace may be used, e.g., as follows:d(A,B)=Σ_(i=1) ^(N)α_(i) |A _(i) −B _(i)|  (Eqn. 5)where A, B denote images and/or image channels, and parameters a; denoteweighs configured to emphasize and/or suppress given channel(s) overother channel(s), N denotes number of channels. By way of anillustration of analyzing an image configured in YUV color space,distance in the luminance channel may be taken with a smaller weightcompared to the distance in the chrominance channels

FIG. 3 is illustrates use of color distance for determining a salientfeature in an image shown in FIG. 2A, according to one or moreimplementations. Distance data shown in FIG. 3 may be determined forexample using a rectilinear (e.g., Manhattan) distance comprising a sumof absolute values of differences between components (e.g., YUV) of agiven pixel and the reference pixel). Contribution of the luminancechannels to the distance may be scaled, e.g., by a factor of 8.Individual channel contributions may be combined using 8-bit saturatingaddition, e.g., configured such that the sum may take maximum value of255. By way of an illustration, sum of 140 and 115 may produce value of255, while sum of 150 and 160 may produce value of 255. The image 300 inFIG. 2A may comprise features of a plurality of colors. Object 302 maycorrespond to an expected target for a given task (e.g., task ofcollecting orange balls, white golf balls, and/or other task). Areference color 308 may be used to determine color distance for pixelsof the image 300. Color reference 308 may comprise a single pixel, or amatrix. In one or more implementations, the distance computation may beeffectuated using Eqn. 5, wherein image A may comprise the image 302,image B may comprise the color reference 308. Image 310 in FIG. 3depicts color distance image determined from the image 300 usingManhattan rectilinear distance determination. Weights α₁ in Eqn. 5 maybe configured as follows to obtain the image 310: luminosity channelweigh may be selected equal 0.125, U,V chromaticity channel weights maybe selected equal 1.0. Area denoted 312 in FIG. 310 may correspond tolowest distance and denote object of interest. In some implementations,salient object detection may be improved by combining color pop-out mask(e.g., the image 210 in FIG. 2A) and the color distance mask (e.g.,image 310 in FIG. 3). In one or more implementations, the combinationmay comprise a pixel-wise multiplication of matrices 210, 310.

Various approaches may be employed in order to determine the referencecolor (e.g., 308 in FIG. 3) used for color distance mask determination.In some implementations, e.g., such as illustrated in FIG. 3, thereference color may be determined as a prevailing (e.g., mean, median,and/or another statistical measure) color in area 212 corresponding tohigh saliency determined using color likelihood analysis, e.g., such asdescribed herein. The reference color determined from, e.g., color mask210 may be utilized for determining the color distance, e.g., presentedin the image 310.

An object may comprise one or more color surfaces (i.e., areas ofsimilar color and reflectance). A color surface may be salient when itis sufficiently distinct from the background and/or surroundingsurfaces. In some implementations, determining a pixel and/or a group ofpixels with the highest value of color saliency may be used foridentifying salient color surface. In some implementations, determiningspatial extent (mask) of the salient color surface may be used fordetermining a statistical parameter of colors (and/or reflectance) inthe salient color surface. The statistical parameter may be used fordetermining of the color distance between the most salient color surfaceand the other ones, and/or to better track the salient object as itmoves in the environment. For example, the mask of the most salientcolor surface may be used for choosing the color prototype used in thecolor distance. In some implementations, the statistical parameter maycomprise prevailing color (e.g., median), least frequently occurringcolor (e.g., below 1% to 5%), and/or other parameter.

In some implementations, the mask of the most salient color surface mayprovide information related to an apparent size of the salient object,and/or a relative distance to the salient object. By way of anillustration, when the area of the color surface increases, it may bemore likely that the object becomes closer to the viewer, or becomesless occluded by other objects.

In order to determine the mask extent (the region of the visual field)occupied by the most salient color surface, the following algorithm maybe used, according to some implementations. Location of the highsaliency area in saliency map (e.g., area of low likelihood (e.g., lessthan 10%) pixels of a given minimum size (e.g., 100 pixels) in FIG. 2A)may be determined. In one or more implementations, the saliency map maybe determined based on color saliency (e.g., image 210 in FIG. 2A), ordetermined by multiplicatively combining multiple saliency maps asdescribed below. In some implementations, saliency map may be blurred,e.g., by using Gaussian blur filter. In order to determine location ofthe salient area (target object), a threshold may be applied to thesaliency map. In one or more implementations, the threshold may comprisea pre-determined value in sample space (e.g., pixels with occurrencecount below 2000 may be deemed as silent; a pre-determined derived value(e.g., least frequently occurring 1% of pixels may be deemed as silent),and/or other threshold approaches. In some implementations, thethreshold level may be determined dynamically based on analysis of oneor more preceding images. In one or more implementations, the thresholdmay be configured to maximize classification success rate and/or ratio(e.g., percent correct ratio of true positives to false positives,and/or other classification performance metrics). In someimplementations, the threshold may be configured based on an evaluationof pixel distribution (e.g., shape of the histogram). In one or moreimplementations, the threshold may be defined as a function of theskewness of the pixel distribution (e.g., histogram). The threshold maybe configured based on a percentile of the distribution, and/or acombination of max( ) operation of a given value and a percentile,and/or yet other factors. In some implementations, the max( ) operationmay be configured to produce an output determined as a maximum value ofa sample comprising (i) a given value (e.g., 1,000,000 pixel counts) and(ii) a percentile value corresponding to the histogram (e.g., 99% valueequal 950,000 pixel counts in some implementations). Location of thetarget may be expressed as a Boolean mask.

In some dynamic saliency threshold implementations, the saliencythreshold may be gradually decreased below the peak value, to produce anincreasingly large contour around the saliency peak. The thresholdlowering may be terminated when the chroma range within the contourexceeds a predetermined numerical value minmax_UV. In YUV colorcoordinates, the chroma range may, for example, be defined asmax(max(U)−min(U), max(V)−min(V)). In YUV coordinates, where U and V maybe between 0 and 255, the allowed minmax_UV may have values between 20and 40. Larger minmax_UV values may allow for a greater color variancewithin a color surface; that is, the algorithm may be more likely totreat adjacent areas of similar chroma as one color surface rather thanas distinct color surfaces.

Tracked objects may be characterized by shape. In some implementations,object circularity analysis may be employed for target detection.Circular shapes may be characterized by symmetries and rotationalinvariances. In some implementations, detection of circular shapes maybe based on Hough moments. In one or more implementations circularityanalysis may comprise the following operations:

-   -   obtain a saliency e.g., using a color pop-out map 210 in FIG.        2A, or via threshold on the color distance (e.g., 310 in FIG.        3), or combination thereof), threshold the saliency map;    -   apply threshold to the saliency to obtain outlines of potential        objects;    -   determine one or more contours of the potential objects using        output of the threshold operation;    -   for individual contour, determine the smallest enclosing circle,        using, e.g., opencv library, described in        http://docs.opencv.org/modules/features2d/doc/common_interfaces_of_feature_detectors.html?highlight=circularity,        the foregoing being incorporated herein by reference in its        entirety;    -   determine a ratio between the area of the contour and the area        of the smallest enclosing circle (this ratio may be referred to        as the circularity ratio r); and    -   apply a transformation to the circularity ratio to obtain a        circularity parameter cp; in some implementations, the        transformation may comprise a power operation (e.g., r²); such        transformation may be used to control the dynamic range of the        circle detection. By way of an illustration, squaring the        circularity ratio r may reduce circularity parameter for less        circular shapes (e.g., wherein cp<1) thereby facilitating        circular shape detection.

Object circularity analysis may be utilized for determining a saliencymap as follows:

-   -   an empty circularity map H may be provided (e.g., an array        initialized with default values, of e.g., 0);    -   determine a plurality of contours using thresholding of a        saliency map (determined using any applicable methodology        including these described herein);    -   for individual contours, determine circularity parameter;    -   sort contours in an increasing order with respect to the        circularity parameter; and    -   for every contour in the original heat map, draw the smallest        enclosing circle of that contour on heat map H. Alternatively        draw the contour itself on the heat map H.    -   The intensity of the fill of the drawn circle (contour) should        be proportional to the circularity parameter.

FIG. 4 illustrates an exemplary circular object detection process,comprising determination of feature circularity, according to one ormore implementations. Operations described with respect to FIG. 4 may beperformed when tracking a ball 174 in FIG. 1 with a robotic vehicle(e.g., 160 in FIG. 1). Image 400 may correspond to an image obtainedwith a camera of the vehicle. Image 400 may comprise 320 by 200 pixelsconfigured using YUV color space and collected at 25 frames per second.The image 400 may be obtained using, e.g., a CMOS sensor comprising 1.5μm generation pixels disposed on a die measuring 4.8 mm×6.1 mm (29.3mm2). The sensor output may be down-sampled to a lower resolution (e.g.,320×200) for processing. Images 400 may be processed online (e.g., inreal time) or stored for off-line processing. It will be appreciated bythose skilled in the arts that above image configuration represents anexemplary implementation, and other image configurations may be realized(e.g., with smaller or greater pixel count and/or pixel size), and/orother alterations. Furthermore, use of smaller pixel size and greaternumber of pixels may correspond to an object representation comprisinggreater number of pixels; use of greater pixel size and smaller numberof pixels may correspond to an object representation comprising fewerpixels. Furthermore, pixel neighborhood dimensions may be adjustedaccordingly with the pixel count and/or size when determining salientarea in an image.

Image 410 may denote motion based saliency map determined from motionanalysis (denoted by arrow 406) between two or more successive images(e.g., 400 and an image taken at another instance in time). Motion-basedsaliency may be configured based on confidence of motion estimation.Different shades in image 410 may denote motion detection confidence.e.g., white area 414 denoting 100%, black denoting 0%, and grey (e.g.,412) denoting an intermediate value, e.g., 0.67).

Image 410 may denote information obtained using a kinematic trackerprocess. By way of an illustration, area denoted by arrow 406 may denotea kinematic prior (e.g., corresponding to a location of an object at aprior time). The kinematic prior may be used to increase probabilityassociated with object occurrence at that location in present image.

Image 420 may denote saliency map determined based on color distancedetermination, e.g., as described above with respect to FIG. 3. Hashedarea 422 may correspond to lower values of color distance compared toarea of solid fill (e.g. 424).

Image 426 may denote saliency map determined based on color likelihoodanalysis (denoted by arrow 408) of the image 402, e.g., color histogramas described above with respect to FIG. 2A. Hashed area 428 maycorrespond to lower values of occurrence (likelihood) of given color(s)compared to area filled with black.

Image 434 may denote saliency map determined based on a combination ofthe color distance map 420 and the color likelihood map 426. In someimplementations, the combination may comprise a pixel-wisemultiplication operation (denoted by arrows 430, 432) between masks 420,426. Hashed area 436 may correspond to greater values of saliencycompared to area of solid fill (e.g. 424).

Image 440 may denote saliency map determined based on contourcircularity analysis of saliency map 434. In some implementations, thecircularity analysis may comprise a threshold operation, determinationof one or more contours, and determination of a ratio between contourarea and the area of the corresponding minimal enclosing circle. Hashedarea 442 in image 440 may denote area of high saliency

In some implementations the identity of the tracked object may beestimated by tracking the motion of elements (features) of the objectfrom its current position to a new position. Motion information maycomprise noise component, particularly in complex environments whereinmultiple moving objects may be present. Motion estimates obtained fromsuccessive image frames (e.g., between 2 and 4 at 25 fps) may provideaccurate object position that may be used as prior information toidentify the object of interest (particularly given a strong saliencymap that filters out much of the background).

In systems equipped with depth sensors (e.g., structured light sensorfor measuring time of flight, stereo camera for determining opticaldisparity, LiDAR sensor, RADAR, ultrasonic, and/or other sensortechnologies capable of providing range to object) distance informationmay be utilized for detecting and/or tracking the object of interest. Agiven a saliency map may comprise multiple areas of high saliency (e.g.,as shown in image 230 in FIG. 2B) corresponding to one or more objects(e.g., 174, 176 in FIG. 1) and/or features. Individual features may bedisposed at different distances from the sensor (e.g., distance betweenobjects 174, 176 and the camera 166 in FIG. 1), and/or distance tofeatures 236, 234 in FIG. 2B. For images obtained in close proximity toone another (e.g., 2 fps to 60 fps for a vehicle speed between 0.01 and2 m/s) distance to a target object may be expected not to changesubstantially between successive images (e.g., for a relative speed of 1m/s between a target and the camera, images acquired 0.1 s apart may beexpected to produce object distance shift of about 0.1 m. Saliency mapmay be combined with the distance information and object may bediscarded from the saliency map based on changes in distance that mayexceed expected distance change range. Combining distance informationwith saliency information may simplify object tracking task and reducenumber of false positives.

The combined saliency information (e.g., multiple saliency mapspertaining to different channels/modalities combined via additive ormultiplicative or hybrid methodology, with additional maps detectingdistinguishing features of the object, e.g. color)) may be used toprovide object location in the image (e.g., location 402 in the image400). This may also be referred to as object tracking or object tracker.

Objects and/or physical environment of the robot may be characterized bycontinuous behavior in time. Therefore, the tracked object may beunlikely to suddenly disappear from its current location and/or reappearat another location. Environment continuity may be exploited byassigning an identity signal close to the previous known location of theobject. In some implementations, involving a kinematic mask, a motionestimate of the object may be determined based on two or more last knownprevious locations to give an estimate of the next location. A saliencymap of expected object location may be provided, which may be integratedwith other sources of information described herein.

Temporal continuity of object and/or environment between time instancest₁, t₂ may be utilized for constructing saliency map S₂ at time instancet₂ based on saliency map S₁ obtained at time instance t1. In someimplementations the saliency map S1 may be combined via a convexcombination with the newly obtained saliency map S₂′ to create the finalmap S₂. In some implementations, the final map S2 may be obtained asfollows:S ₂ =αS ₂′+(1−α)S ₁.Such methodology allows to mitigate certain transient noise (e.g.thermal camera noise) at a cost of providing lower signal in rapidlychanging environments (where frame to frame true variability of theenvironment is large, e.g. in a very fast-moving robot).

In some implementations, histogram back projection may be used toprovide evidence for the object tracker. As used herein the term trackermay be used to describe a process realized on a computer and/or otherdigital and/or analog hardware configured to produce, based on analysisof an image, a set of coordinates reflecting a position of an object inthe image and/or object coordinates referenced to a sensing field of theimage sensor (e.g., camera view field, sonar/radar sensing area.

A histogram of color channel (or multiple channels, or channels encodingother features than color) may be determined in the area where theobject is present. In subsequent steps, a heat map may be generated byback-projecting the counts of the bins back into the pixel space (thatis every pixel in the heat map is proportional to bin count to which theoriginal pixel in the image belongs). In some implementations, thehistogram back projection may be determined only on chromatic channels(containing color information) to make the method tolerant to luminancechanges. In some implementations back projection may be performedindependently for individual chromaticity (or other feature) channels.In some implementations, the resulting heat map may be obtained byadditive, multiplicative, or hybrid integration (e.g., as describedbelow).

In some implementations, spatial constraints may be provided as an inputinto the tracker. Use of prior spatial information may substantially(e.g., by eliminating 50% or more of false positives) improve trackingperformance. For example, in some applications a given location withinthe sensing field (e.g. image in camera sensor implementations) may becharacterized by frequent presence of objects other than the object ofinterest. These extraneous objects/features may also be referred to asdistractors. The object of interest may rarely (relative to thedistractors) appear at the given location. In such case, a prior spatialprobability of appearance map may be determined and incorporated intothe tracking process in order to reduce the likelihood of false objectdetection at the given location. In some implementations, the priorspatial map may be used to enhance object detection on one or moreareas.

In some implementations, a pattern may exist in a spatial area where theobject of interest may appear. Such pattern (e.g., green lawn) may beknown in advance in some applications, while in some applications (e.g.,location of a road in an image, location of vehicles of a given color ina mining pit) may not be known in advance. The tracking process mayintegrate information about where the objects were detected to enhancegenerate additional prior saliency map. Incorporation of the priorsaliency map into object tracking process, may reduce the number offalse positives in the area where the object of interest may be lesslikely to appear.

Some existing implementations of computer vision trackers use user input(e.g., via mouse/keyboard) in order to provide an initial target and/ortarget location (e.g., initial bounding box around the tracked object).Such approaches may not be well suited for autonomous operation, e.g.,autonomous vehicles, and operating devices (e.g., robotic appliances)that may not be equipped with a screen and input device.

According to one or more implementations, object tracker of thedisclosure may use the saliency mask in order to locate an object ofinterest. The saliency mask may be used to prime/initialize one or moreobject trackers. In some implementations, the uninitialized tracker maybe configured to determine the saliency map using one or more approachesdescribed herein (e.g., color saliency map). The uninitialized trackermay identify one or more candidate objects (salient areas) and select atarget object. In some implementations, the target object may beselected based on largest area of given saliency (e.g. greater than90%), and are containing highest saliency (peak area), are containingmost saturated color, and/or a weighted combination of multipleselection criteria.

Once the initial target area is selected, additional tracker parametersmay be configured. A color reference (e.g., 308 in FIG. 3) for colordistance determination may be selected. In some implementations, one ormore individual colors that may occur within the target area may betested for discriminability of determining the target versus backgroundusing color. The discriminability may be determined using color distanceabove a given threshold. A given color from the target area whichmaximizes discriminability (e.g., greatest color distance to thebackground) may be selected as the reference color.

Further, once the reference color has been chosen, color distancethreshold may be determined. In some implementations, a plurality ofcolor distance maps may be determined for a plurality of referencecolor/threshold combinations. The color/threshold combinationcorresponding to greatest discriminability (e.g., greater cumulativeover a given area color distance to a reverence color) may be selected.

A variety of approaches may be used for determining initial target area.In some implementations, a combination of colors and/or other featuresmay be used. By way of an illustration, motion saliency map may be usedto determine the initial target area. To facilitate motion based trackerinitialization, the object of interest may be moved back and forth infront of the camera. In some implementations, a distance saliency mapmay be used to prime the tracker, in which case the object asubstantially different distance (e.g., closer) than others may beselected. In some implementations, a weighted combination of a pluralityof saliency masks may be used to select the most salient object fortracking.

In some implementations, the identity of the tracked salient object maybe estimated via computation of color distance, between the pixelsrepresenting the salient object and a previously initialized colorprototype.

In some implementations, the saliency mask representing a number ofsalient objects in the scene may be modified morphologically to connectclose by components which are likely representing a single salientobject (e.g. a colorful ball, obstructed by a black cable whichsubdivides the salient region into two close by regions). The amount ofsuch morphological connection may be such that pixels closer than 2% ofthe larger image dimension will be connected.

A number of colors may be selected in order to track the identity of anobject. In some implementations, a clustering algorithm may be employedto determine one or more colors (e.g., a set of colors) that maycorrespond to the object. In one or more implementations, individualcolors for the color set may be selected based on a distance from agiven color (e.g., most distant colors within a color space from thebackground). In some implementations, colors with saturation value abovea threshold (e.g., greater than 70% saturation) may be selected. In someimplementation, two or more color selection methods may be combined,e.g., the clustering technique may be applied to output of colorsaturation analysis and/or color distance analysis in order to determinecolors for the set of colors used in tracking.

In some implementations, the color distance to multiple color prototypesmay be computed for every contour (or a subset thereof) in the saliencymask. The likelihood that the salient object is that which should betracked may be estimated as inversely proportional to the max( )operation over minimal distances over the contour, for one or morecolors in consideration:P(c)˜1/dmax; dmax=max(min(d _(i))),  (Eqn. 6)where d_(i) is the color distance for a given pixel within the contour.

For example, if one of the distances to color prototypes for a givencontour is larger than distances associated with other contours (e.g.such color is missing from the object), it may be picked by the max( )operation and may lower the likelihood of the object detection for thegiven contour. By way of an illustration, object detection in contour C1and contour C2 may be performed as follows. A minimum color distance d1of pixels within the contour C1 to a reference color in 3-channel colorspace may comprise d1={10, 15, 12}; minimum color distance d2 of pixelswithin the contour C2 to the reference color in 3-channel color spacemay comprise d2={20, 10, 50}. Maximum operation applied to set d1 mayproduce a value of 15; maximum operation applied to set d2 may produce avalue of 50. In accordance with Eqn. 6, likelihood of detection of theobject in the contour C2 may be reduced relative likelihood of detectingthe object in the contour C1. It will be recognized by those skilled inthe arts that max( ) selection operation comprises one exemplaryimplementation. Other means of combining individual distances may beused consistent with the present disclosure, e.g. mean, median,Euclidean distance, L1 norm, or combinations of the foregoing.

In some implementations, a number of colors not belonging to the targetmay be acquired in the priming phase to provide negative evidence forobject detection. In such case, the detection of one of those colorswithin the saliency area will decrease the likelihood of that area beingdetermined as the object of interest.

In some implementations, the “negative” colors may be sampled randomlyunder the constraint of being as far as possible in the color space topositive colors examples (those detected on the priming object). In someimplementations, the term positive color may be used to describe colorsthat may correspond to the object with high probability (e.g., greaterthan 90%). The term negative color may be used to describe colors thatmay be absent from the object with high probability (e.g., greater than90%).

In some implementations, the negative colors may be selected from thebackground (non salient area of the priming image), to maximizediscriminability between the object and the background.

In some implementations, a single parameter may be computed to indicatethe distance to all the negative example colors. In someimplementations, that parameter may be computed as the max of min, mean,median, Euclidean distance, L1 norm, and/or any combination thereof.

In some implementations, the negative evidence parameter may besubtracted from the positive evidence parameter to determine thelikelihood of object detection. In other implementations, a set oflogical statements and thresholds may be used to determine objectdetection likelihood. By way of an illustration, negative colorinformation may be combined with the contour distance information asfollows. A minimum color distance d1 of pixels within the contour C1 toa reference color in 3-channel color space may comprise d1={10, 15, 12};minimum color distance d2 of pixels within the contour C2 to thereference color in 3-channel color space may comprise d2={20, 10, 50}.Distance to a color that is deemed as absent from the object (negativecolor) may be determined as 5 for the contour C1 and 100 for the contourC2. A difference operation may be used to combine the positive colordistance and the negative color distance to determine likelihood of theobject detection as follows:P2(c)˜min(dmax^(j) −dneg^(j)); dmax=max(min(d _(i))),  (Eqn. 7)where dmax^(j) denotes largest of minimum color distances for contour j,and dneg^(j) denotes smallest negative color distance for contour j.The difference term of Eqn. 7, determined for contour C1 is 10, whiledifference term of Eqn. 7, determined for contour C1 is −50. Accordingto Eqn. 7, the likelihood of the detecting the object at the contour C2may be increased relative the likelihood of the detecting the object atthe contour C1.

FIG. 5 illustrates use of a saliency map configured based on colorlikelihood analysis of an image as the priming map for detecting salientfeature, according to one or more implementations. Map 500 in FIG. 5 maybe obtained based on analysis of likelihood of occurrence of pixel colorin an image (e.g., image 200 in FIG. 2A). Map 500 may be obtained usinghistogram operations described above with respect to image 210 of FIG.2A, in some implementations. Saliency may be configured inverselyproportional to frequency of occurrence of a given color in the image200. The map 500 may comprise one or more areas of high saliency (e.g.,the area 502). The map 500 may comprise areas of intermediate saliency(e.g., the gray areas 504). In order to increase signal to noise ratio,and/or reduce probability of false detections, the map 500 may bespatially filtered in order to reduce and/or altogether remove smallerareas of low saliency. In some implementations, the filter may comprisea Gaussian 2-dimensional mask of 9×9 pixels for an image of 320×200pixels. A threshold may be applied to the filtered map (or to the map500 in some implementations) in order to remove areas of low saliency.In some implementations, the threshold may be configured to remove areaswith saliency measure below 70%. As may be seen from map 510 in FIG. 5,the high saliency area (502 in map 510) is preserved throughfiltering/thresholding operations, while smaller areas of lower saliency(504 in map 500) are absent in the map 510 thereby improving reliabilityof detection of the salient area 512. The map 510 may be utilized fordetermining the reference color, which provides greatestdistinguishability of the target object from the surrounding background.

Saliency maps (obtained using a variety of approaches) may be combinedin order to improve detectability of an object/feature. Detectabilityimprovement may be characterized by, e.g., a reduced number of falsepositives (e.g., an area deemed as silent that does not contain thetarget object) by the object detection process.

In some implementations, two or more saliency maps may be combinedmultiplicatively to produce a combined map representing the likelihoodof presence of the object of interest. Individual maps H_(i) may becharacterized by a scalar parameter c_(i) signifying the confidence ofthe respective map. Given confidence c_(i)∈[0,1], saliency maps Si maybe combined multiplicatively as follows:C=Π _(i=1) ^(n)[(1−c _(i))+S _(i) c _(i)]  (Eqn. 8)where S_(i)(x,y)∈[0,1]. In one or more implementations a saliency mapcharacterized by low confidence (e.g., c=0) may contribute a constantequal to 1 in the product of Eqn. 8, therefore may not change therelative saliency of one area over another area in the resultant map C.A saliency map characterized by greater confidence (e.g., c=1) maycontribute substantially to the resultant map C. In someimplementations, a saliency map may be obtained as function of anothersaliency map. By way of an illustration, a map determined based oncircularity analysis (e.g., as described above) may be obtained based onthe color saliency map.

FIG. 6 illustrates use of multiplicative combination of multiplesaliency maps for detecting a target object/feature according to one ormore implementations. Individual saliency maps may be obtained using avariety of methods, including these described herein. Saliency maps 600,610, 620 shown in FIG. 6 may be obtained using, e.g., pixel occurrencelikelihood analysis (e.g., described with respect to FIG. 2A), usingmotion analysis (e.g., described with respect to map 410 of FIG. 4, andusing color distance analysis, such as e.g., described with respect tomap 310 of FIG. 3 above. The maps 600, 610, 620 may be combinedmultiplicatively, e.g., using formulation of Eqn. 8 depicted by arrow618 in FIG. 6. The combined saliency map 630 may be characterized by anincreased signal-to-noise (increased saliency) associated with the areacomprising 632 the object of interest relative its surroundings (e.g.,634) compared to the saliency of the areas 602, 612, 622 in the maps600, 610, 620, respectively.

In one or more implementations, saliency map C may be determined basedon a weighted sum (e.g., weighted average) of two or more individualsaliency maps S_(i) as follows:C=Σ _(i=1) ^(n)α_(i) S _(i)  (Eqn. 9)where α_(i) denote weighting parameters.

In certain applications, a fusion of additive (e.g., of Eqn. 9) andmultiplicative (Eqn. 8) saliency map combination techniques may be used.Certain saliency maps may provide capability to exclude certainlocations in the image from being considered as containing the object.For example, if it may be known that the object of interest may morelikely reside no farther than a given distance D from the device (e.g.,the camera 166 in FIG. 1) a range-based saliency map, thresholded at thegiven distance D, may be integrated multiplicatively with other maps.Setting map locations (corresponding to ranges further than the distanceD) within the distance-based saliency map to zero may be used to ensurethat false positives that may arise in other methods of saliencyanalysis (e.g., based on color) may not bias the final saliencydetermination outcome. Such approach may be referred to as “veto” and/ora “filter out” operation.

In some implementations wherein evidence of presence/lack of feature maynot be sufficiently high to exercise the veto operation (e.g., while theobserved color may not be sufficiently close to the target object due tochanges in illumination), the area in question may still be consideredas a possible location of the target object, one or more saliency mapsmay be combined additively. In some implementations, the overallsaliency determination approach may comprise a combination of additiveand multiplicative approaches.

Output of saliency determination operation (e.g., combined saliency map)may be used to determine the target object location and/or bounding boxfor tracking the object. Various techniques and heuristics may be usedin order to determine whether the tracker may still be following theobject of interest. A bounding box may comprise the smallest rectangleenclosing the contour (area) where the object may be detected. Boundingbox location (e.g., center coordinates) may be used by the trackingprocess to determine object location; bounding box dimensions may beused by the tracking process to determine object size. In someimplementations, bounding box may be used for subsequent processing ofimages (e.g. by a machine learning system).

The bounding box may be determined in a variety of ways. In someimplementations, a flood-fill on the color distance originating at thepeak of the combined saliency map with a tolerance (e.g. referring toopencv documentations of flood fillhttp://docs.opencv.org/modules/imgproc/doc/miscellaneous_transformations.html,parameter loDiff=10) may be performed to estimate a connected component(an area in the image enclosed by a single contour) containing the peakof the combined saliency. The term “fill operation tolerance” may beused to describe image area fill operation wherein the fill process mayassign given pixel values (e.g., given color) to pixels within an areauntil a difference across two neighboring pixels may be encountered thatis greater than the tolerance.

Location and/or dimensions of the bounding box may be determined bydetermining the smallest enclosing rectangle of the connected component.In some implementations, the flood-fill may be determined on thesaliency map determined based on analysis of color likelihood (e.g., 210in FIG. 2A). In some implementations, the flood-fill may be determinedusing a saliency map obtained based in two or more additively combinedsaliency maps Si using one or more image characteristics than onemodality (for example color distance, color likelihood, and/or othercharacteristics). Individual maps Si may be thresholded prior to beingcombined. In some implementations, the connected component may bedetermined using the combined (multiplicative) saliency map. Suchapproach may enable tracking of objects that may not be characterized bya distinctive (salient) color compared to the background so thattherefore the color distance alone may be providing a weak (e.g., below50% of the maximal) saliency signal.

In some implementations, the tracked object may contain multiple colors.Presence of multiple colors in close vicinity to one another maysimplify tracking. In some implementations, the initial area of interestas provided by the saliency map (pop-out mask) may be analyzed and anumber of distant (in color space) color prototypes may be acquired. Forindividual color prototypes, a color distance mask may be determined forsubsequent frames. For a given new frame, a set of candidate objectlocations may be established based on the saliency mask. For individualcandidate regions, a set of minimal distances to the color prototypesmay be determined. A logical analysis may be performed in order todetermine an area that may produce the best match. In one instance thebest match area may be selected based on color distances for a givenarea being within a given range (e.g., below a given threshold). In somecases, the color distance range may be configured individual colorprototypes, e.g., being a function of color discriminability from thefirst (priming frame). In some implementations, a weighted sum ofminimal color distances may be determined and compared to a threshold.In such case even if one of the colors from the prototype set maymissing from the area of interest (large distance), the presence ofother colors from the prototype set may cause the area to be selected asa possible location of the target.

In some cases multiple objects that may satisfy the tracking criteriamay occur in an image. In one or more implementations, the trackingprocess may return positions and/or bounding boxes of all of thelocations associated with these objects, may return the position of thelargest region of interest; and/or may return the region closest to thepreviously identified object location. In some cases, the trackingprocess may produce a saliency describing positions and likelihoods ofthe target object appearing in the scene. In some implementations, suchraw output may be forwarded to machine learning system; e.g., onetrained to classify the presence of object of interest based on one ormore factors such as e.g., pre-recorded and tagged set(s) of movies.

The output of the tracking process may be used to drive robotic deviceto execute a task (e.g., approach object 174 in FIG. 1 and/or land onthe pad 1012 in FIG. 10). The output of the tracking process may be usedas an input to a system configured to predict next motor action of arobot (referred to as predictor). Various ways of representing thetracker output may be used for that application, including, e.g., rawsaliency map described below, normalized parameters of the bounding box.

In one or more implementations, the tracking process may be configuredto determine if the object of interest may still present in a givenimage (e.g., 200 in FIG. 2A). If the determination indicates that theobject may be present, the tracking process may communicate informationrelated to the bounding box of the object in visual space coordinates.In some implementations, target object location (e.g., 402 in FIG. 4)referenced to the image may be combined with image sensor parameters(e.g., orientation in 3D, position) in order to determine bounding boxlocation in visual space coordinates. If the object is absent, an objectnot found indication (e.g., a bounding box with negative coordinatesand/or a null object) may be provided. Before entering the predictor(which for example maybe implemented a neural network having a pluralityof input units) the tracker output may be converted to a form moresuitable for subsequent processing. In some implementations, the actualbounding box may be converted to a saliency mask comprising zero valuesoutside the box and values of one inside the box. Such mask may bepassed through a preprocessing layer of neurons having spatial receptivefields covering the image space. Individual neurons may be configured toproduce output configured based on the fraction of the receptive fieldoccupied by the box (value one). A neuron which has only zero values inits receptive field may not produce output (activation zero). A neuronwhose entire receptive field is occupied by the box (value one) may befully activated. The vector of such activations projections ontoreceptive fields) may be provided as an input to the predictor. In someimplementations, the center coordinates of the bounding box, togetherwith its width and height may be provided as inputs to.

The predictor may comprise a learning process configured to determineassociations between sensory context and action by the robot. In one ormore implementations, the context may comprise one or more of robotstate (e.g., location or motion information, (position, orientation,speed), platform state or configuration (e.g., manipulator size and/orposition), available power and/or other), state of the environment(e.g., object size, location), environmental state (wind, rain),previous state information (e.g., based on historic states of robotmotions), and/or other characteristic state information. In someimplementations, the predictor may comprise an artificial neuron networkconfigured to implement e.g., a perceptron, a random k-nearest neighborsclassification process, a look up table learning process and/or otherimplementations, including e.g., such as described in U.S. patentapplication Ser. No. 14/607,018 entitled “APPARATUS AND METHODS FORTRAINING PATH NAVIGATION BY ROBOTS”, filed Jan. 27, 2015, Ser. No.14/588,168 entitled “APPARATUS AND METHODS FOR TRAINING OF ROBOTS”,filed Dec. 31, 2014, Ser. No. 14/244,890 entitled “APPARATUS AND METHODSFOR REMOTELY CONTROLLING ROBOTIC DEVICES”, filed Apr. 3, 2014, Ser. No.13/918,338 entitled “ROBOTIC TRAINING APPARATUS AND METHODS”, filed Jun.14, 2013, U.S. patent application Ser. No. 13/918,298 entitled“HIERARCHICAL ROBOTIC CONTROLLER APPARATUS” filed Jun. 14, 2013, Ser.No. 13/907,734 entitled “ADAPTIVE ROBOTIC INTERFACE APPARATUS ANDMETHODS”, filed May 31, 2013, Ser. No. 13/842,530 entitled “ADAPTIVEPREDICTOR APPARATUS AND METHODS”, filed Mar. 15, 2013, Ser. No.13/842,562 entitled “ADAPTIVE PREDICTOR APPARATUS AND METHODS FORROBOTIC CONTROL”, filed Mar. 15, 2013, Ser. No. 13/842,616 entitled“ROBOTIC APPARATUS AND METHODS FOR DEVELOPING A HIERARCHY OF MOTORPRIMITIVES”, filed Mar. 15, 2013, Ser. No. 13/842,647 entitled“MULTICHANNEL ROBOTIC CONTROLLER APPARATUS AND METHODS”, filed Mar. 15,2013, and Ser. No. 13/842,583 entitled “APPARATUS AND METHODS FORTRAINING OF ROBOTIC DEVICES”, filed Mar. 15, 2013, each of the foregoingbeing incorporated herein by reference in its entirety.

In some implementations, process of determining as to whether thetracker may be following the object of interest may be incorporated into the behavior prediction process. In this case the integrated system(robot together with behavior predicting process) may learn thebehaviors relevant for the “tracked” object. In some instances a finalheat map of object presence may be passed directly to the predictivesystem. In some implementations, the heat map may be passed through apreprocessing layer (e.g., as described in the paragraph above). In someimplementations, the location of the peak of the heat map, together withthe value at the peak and an estimate of the size of the peak at half ofthe maximal value, may be passed to the predictive system.

By avoiding the classification (decision if the object is tracked ornot) the predictive process may take into account additional informationnot available to the tracker on whether the final saliency map maycontain behaviorally relevant information. For example, the predictiveprocess may learn to adjust the threshold for the detection based on oneor more of the position/orientation of the robot, time of day, and/orother aspects that may be inaccessible to the tracker process.

Parts of the above described methodology may be applicable to a varietyof signal modalities, e.g., in thermography, sonar data, radar data,and/or other applications. Saliency detection may be carried out insonography, for example, by determining relevant histograms on thespectral representation of the time slice of sound.

Various computerized devices may be employed for implementingmethodology described herein. One particular implementation of thecomputerized processing system, for use with the object detection,tracking, and/or control output generation methodology of the disclosureis illustrated in FIG. 7.

FIG. 7 illustrates a computerized system comprising a signal processingapparatus of the disclosure, in accordance with one implementation. Thesystem 700 may comprise a computerized entity 706 configured tocommunicate with one or more signal processing devices 710 (e.g., 710_1,710_2). In some implementations, the entity 706 may comprise a computingcloud entity (e.g., a cloud service, a server, in a public, private orhybrid network). In one or more implementations, the entity may comprisea computer server, a desktop, and/or another computing platform that maybe accessible to a user of the controller 710. In some implementationsof the cloud computing services, one or more signal processing devices710 may communicate with the entity 706 in order to access computingresources (e.g., processing cycles and/or memory) in order to, e.g.,detect features and/or objects in sensory data provided by, e.g., sensormodule 172 of control system in FIG. 1. In some implementations, thesignal processing devices 710 may communicate with the entity 706 inorder to save, load, and/or update, their processing configuration(e.g., processing configuration 812 in FIG. 8). The configuration maycomprise executable code (e.g., binary image files), bytecode, an arrayof weights for an artificial neuron network (ANN), and/or other computerformats. In some implementations, the signal processing device 710 maycommunicate with the entity 706 in order to save, and/or retrievelearned associations between sensory context and actions of a robot,e.g., as described in U.S. patent application Ser. No. 14/244,888,entitled “LEARNING APPARATUS AND METHODS FOR CONTROL OF ROBOTIC DEVICESVIA SPOOFING”, filed Apr. 3, 2014, now issued as U.S. Pat. No.9,630,317, and incorporated herein by reference in its entirety.

In FIG. 7, one or more signal processing devices (e.g., 710_1) mayconnect to the entity 706 via a remote link 714, e.g., Wi-Fi, and/orcellular data network. In some implementations, one or more signalprocessing device (e.g., 710_2) may connect to the entity 706 via alocal computerized interface device 704 using a local link 708. In oneor more implementations, the local link 708 may comprise a network(Ethernet), wireless link (e.g. Wi-Fi, Bluetooth, infrared, radio),serial bus link (e.g., USB, Firewire) and/or other. The localcomputerized interface device 704 may communicate with the cloud serverentity 706 via link 712. In one or more implementations, links 712and/or 714 may comprise an internet connection, and/or other networkconnection effectuated via any of the applicable wired and/or wirelesstechnologies (e.g., Ethernet, Wi-Fi, LTE, CDMA, GSM, and/other).

In one or more applications that may require computational power inexcess of that that may be provided by a processing module of the signalprocessing device 710_2 the local computerized interface device 704 maybe used to perform computations associated with training and/oroperation of the robotic body coupled to the signal processing device710_2. The local computerized interface device 704 may comprise avariety of computing devices including, for example, a desktop PC, alaptop, a notebook, a tablet, a phablet, a smartphone (e.g., aniPhone®), a printed circuit board and/or a system on a chip (SOC)comprising one or more of general processor unit (GPU), fieldprogrammable gate array (FPGA), multi-core central processing unit(CPU), an application specific integrated circuit (ASIC), and/or othercomputational hardware.

FIG. 8 is a functional block diagram detailing components of a signalprocessing device (e.g., 710 of FIG. 7) in accordance with oneimplementation. The apparatus 800 may comprise a memory component 812configured to store a processing configuration and/or processing code.In one or more implementations, the processing configuration maycomprise object color, size, shape, thresholds, weight coefficients,and/or other parameters. Additional memory 814 and processing capacity816 is available for other hardware/firmware/software needs of theapparatus. The processing component 816 may interface to the sensorycomponent 820 in order to obtain and process sensory data for, e.g.,object detection, face tracking, stereo vision, and/or other tasks.

The processing component may interface with the mechanical 818, sensory820, electrical 822, power 824, and user interface 818 components viadriver interfaces and/or software abstraction layers. Additionalprocessing and memory capacity may be used to support these processes.It will be appreciated that these components may be fully controlled bythe robotic brain. The memory and processing capacity may aid inmanagement of processing configuration (e.g., loading, replacement),operations during a startup, and/or other operations. Consistent withthe present disclosure, the various components of the device may beremotely disposed from one another, and/or aggregated. For example, thelearning configuration may be executed on a server apparatus, andcontrol the mechanical components via network or radio connection whilememory or storage capacity may be integrated into the brain. Multiplemechanical, sensory, or electrical units may be controlled be a learningapparatus via network/radio connectivity.

The mechanical components 818 may include virtually any type of devicecapable of motion or performance of a desired function or task. Thesemay include, without limitation, motors, servos, pumps, hydraulics,pneumatics, stepper motors, rotational plates, micro-electro-mechanicaldevices (MEMS), electroactive polymers, and/or other mechanicalcomponents.

The user interface components 818 may comprise virtually any type ofdevice capable of registering inputs from and/or communicating outputsto a user. These may include, without limitation, display, touch,proximity sensitive interface, light, sound receiving/emitting devices,wired/wireless input devices and/or other devices.

The sensory devices 820 may enable the processing apparatus to acceptsensory input from external entities. These may include, withoutlimitation, video, audio, capacitive, radio, vibrational, ultrasonic,infrared, and temperature sensors radar, lidar and/or sonar, and/orother sensory devices.

The electrical components 822 may include virtually any electricaldevice for interaction and manipulation of the outside world. This mayinclude, without limitation, light/radiation generating devices (e.g.LEDs, IR sources, light bulbs, and/or other devices), audio devices,monitors/displays, switches, heaters, coolers, ultrasound transducers,lasers, and/or other electrical components. These devices may enable awide array of applications for the robotic apparatus in industrial,hobbyist, building management, medical device, military/intelligence,and other fields (as discussed below).

The communications interface 826 may include one or more connections toexternal computerized devices to allow for, inter glia, management ofthe processing and/or robotic device, e.g., as described above withrespect to FIG. 1 and/or with respect to FIG. 7. The connections mayinclude any of the wireless or wireline interfaces discussed above, andfurther may include customized or proprietary connections for specificapplications. In some implementations, the communications interface 826may comprise a component (e.g., a dongle), comprising an infraredsensor, a radio frequency antenna, ultrasonic transducer, and/or othercommunications interfaces. In one or more implementation, thecommunications interface may comprise a local (e.g., Bluetooth, Wi-Fi)and/or broad range (e.g., cellular LTE) communications interfaceconfigured to enable communications between the processing apparatus(e.g., 710 in FIG. 7) and a remote computing entity (e.g., 706, 704 inFIG. 7).

The power system 824 may be tailored to the needs of the application ofthe device. For example, for a small-sized lower power appliance, awireless power solution (e.g. battery, solar cell, inductive(contactless) power source, rectification, and/or other) may beappropriate. For building management applications, battery backup/directwall power may be superior. In addition, in some implementations, thepower system may be adaptable with respect to the training of theappliance apparatus (e.g., such as described in U.S. patent applicationSer. No. 14/489,242, entitled “APPARATUS AND METHODS FOR REMOTELYCONTROLLING ROBOTIC DEVICES”, filed Sep. 17, 2014 and incorporatedherein by reference in its entirety). The training may improveoperational efficiency of the appliance (to include power consumptionefficiency) through learned management techniques specifically tailoredto the tasks performed by the appliance apparatus.

FIGS. 9A-9D illustrate gestures of a human operator used forcommunicating control indications to a robotic device comprising featuredetermination apparatus described herein, in accordance with one or moreimplementations. The robotic device may comprise, e.g., the apparatus160 comprising a camera 166. Objects and/or features shown in FIGS.9A-9D may be obtained by a controller of the robot during operationusing object/feature detection based on saliency analysis of videoimages provided by the camera.

FIG. 9A is a top view of a user and may illustrate a base posture 920 ofthe user. FIG. 9B may depict user gestures 900 communicating a rightturn action to a robotic device (e.g., the vehicle 160 in FIG. 1. Therobotic device 160 may utilize images provided by the camera 166 inorder to detect position of the user arms 908, 908. In someimplementations, the arm 908, 904 position may be determined usingsaliency determination methodology configured based on color distance,color likelihood, distance, motion and/or other approaches describedherein. By way of an illustration, the gesture in FIG. 9B may bedetermining based on a comparison of distance between the robot and theuser antis in positions 604, 608 in FIG. 9B relative the user arms inposition 924 in FIG. 9A. In one or more implementations, the gesture inFIG. 9B may be determining based on a comparison of distance between therobot and the user arms in positions 904, 908 relative the user head 902in FIG. 9B.

FIG. 9C is a side view of the user and may depict user gesture 910communicating a stop action to a robotic device (e.g., the vehicle 160in FIG. 1). A processing apparatus may utilize images provided by thecamera 166 in order to detect position of the user arms, head 942, 912,and/or hands 914, 944. In some implementations, the hand 942, 944position may be determined using the saliency determination methodologydescribed herein. By way of an illustration, the gesture in FIG. 9C maybe obtained based on a comparison of color between the robot and theuser hands in position 914 in FIG. 9C relative the user hand in position944 in FIG. 9D. In one or more implementations, the gesture in FIG. 9Cmay be determined based on a comparison of distance between the robotand the user hand in position 914 relative the user head 912 in FIG. 9Canalysis of arm 914 motion, and/or other approaches. In someimplementations (not shown) the user may communicate an indication tothe robotic device by, e.g., appearing in view of the camera. By way ofan illustrating, the user stepping in front of the vehicle may indicateto the vehicle a stop action.

FIG. 10 illustrates use of saliency and/or object detection methodologyby an unmanned robotic apparatus configured for autonomous navigation,in accordance with one or more implementations.

The unmanned autonomous vehicle (AUV) 1000 may comprise one or morecameras 1002 disposed spatially from one another. Images provided by thecamera 1002 may be analyzed using any applicable saliency detectionmethodology described herein (e.g., with respect to FIGS. 2A-6). Theanalysis may provide information related to presence and/or location ofobstacles (e.g., the 1010 based on color distance saliency map),location of the landing pad 1012, and/or other objects. Controller ofthe device 1000 may utilize the saliency information and/or vehiclemotion information to control actuators 1004 when landing, duringtake-off and or navigating around obstacles.

In some implementations, video streams provided by individual ones ofthe two or more cameras 1002 may be interleaved and encoded using, e.g.,methodology described in U.S. patent application Ser. No. 14/326,374entitled “APPARATUS AND METHODS FOR DISTANCE ESTIMATION USING STEREOIMAGERY”, filed Jul. 8, 2014, the foregoing being incorporated herein byreference in its entirety. The encoding may enable controller of thevehicle 1000 to determine distance 1006 between the vehicle 1000 and thelanding location 1012, and/or distance 1018 to obstacles (e.g., 1010).The controller may utilize the distance, the saliency information,and/or vehicle motion information to control actuators 1004 whenlanding, during take-off and or navigating around obstacles.

FIGS. 11-13 illustrate exemplary methods of determining saliency insensory input, according to one or more implementations of thedisclosure. The operations of methods 1100, 1200, 1300 presented beloware intended to be illustrative. In some implementations, methods 1100,1200, 1300 may be accomplished with one or more additional operationsnot described, and/or without one or more of the operations discussed.Additionally, the order in which the operations of methods 1100, 1200,1300 are illustrated in FIGS. 11-13 described below is not intended tobe limiting.

Methods 1100, 1200, 1300 may be implemented in one or more processingdevices (e.g., a digital processor, an analog processor, a digitalcircuit designed to process information, an analog circuit designed toprocess information, a state machine, and/or other mechanism forelectronically processing information and/or configured to executecomputer program modules stored as computer readable instructions). Theone or more processing devices may include one or more devices executingsome or all of the operations of methods 1100, 1200, 130 in response toinstructions stored electronically on a non-transitory electronicstorage medium. The one or more processing devices may include one ormore devices configured through hardware, firmware, and/or software tobe specifically designed for execution of one or more of the operationsof methods 1100, 1200, 1300. The operations of methods 1200, 1220, 1240may be implemented by a computerized apparatus (e.g., 710_1, 706, 800 inFIGS. 7-8) configured to control a robotic device (e.g., 160 in FIG. 1).

FIG. 11 illustrates a method of using saliency detection for tracking ofobjects, in accordance with one or more implementations.

At operation 1102 of method 1100 saliency map may be determined.Saliency map determination may be configured based on analysis of one ormore characteristics of an image (e.g., the image 200 in FIG. 2A). Thesaliency map be configured based on a determination of a deviation ofthe characteristic in one portion of the image relative other portions.The characteristic may comprise pixel color for one or more channels(e.g., YUV), pixel color distance to a reference color, orientation ofone or more groups of pixels, motion magnitude and/or direction for agroup of pixels, and/or other characteristics. By way of anillustration, likelihood of pixel value occurrence in the image may bedetermined. Saliency of a pixel at a location within the saliency mapmay be determined based on the likelihood value pixel at that location.In some implementations, saliency map may comprise multiple components,e.g., map component determined based on likelihood, map componentdetermined based on color distance, and/or other approaches.

At operation 1104 saliency map may be analyzed to determine an area ofhigh saliency for a given task. Level of saliency considered high may beconfigured based on a specific task. By way of an illustration, in anapplication of locating a target object of a given color on abackground, an area with color occurrence below 5% may be deemed assalient. In some implementations, the saliency map analysis may comprisedetermination of peak saliency, area around the peak a contour of agiven level (e.g., threshold), largest contiguous area at or above giventhreshold; a contiguous area at or above given threshold, and/or otheroperations. Operation 1104 may produce one or more areas characterizedby high saliency.

At operation 1106 one or more high saliency areas determined atoperation 1104 may be analyzed. Analysis of a given salient area maycomprise detecting presence of a particular color or set of colors,determining absence of a given color or a set of colors, presence orabsence of features characterized by, e.g., high frequency content(e.g., edges), determining mean orientation, and/or other features.Operation 1106 may be configured to produce information related topresence of the object of interest in one or more areas of highsaliency.

At operation 1108 information related to presence of the object ofinterest may be provided to a tracking process. In some implementations,the information may comprise object location, color, shape, bounding boxsize and/or location. Operation 1108 may be referred to as trackerpriming wherein information about presence and/or location of an objectobtained using one image characteristic (e.g., color distance) may beutilized to initialize and/or assist a tracking process configured tooperate based on another characteristic (e.g., motion).

FIG. 12 illustrates a method of determining a saliency map based onlikelihood information, in accordance with one or more implementations.

At operation 1202 of method 1200 a histogram of image pixel values maybe determined for one or more channels of an image. In someimplementations the image may comprise an image in YUV color spacecomprising 320×240 pixel matrix for Y luminosity channel, 160×120 pixelmatrix for individual ones of the U,V chromaticity channels. Pixelvalues for individual (YUV) channels may be characterized by 8-bitresolution thereby corresponding to pixel value range between 0 and 255.The image may be characterized by a 3-dimensional color space (e.g.,cube 206 in FIG. 2A). The color space may be partitioned into one ormore bins, e.g., luminosity channel may be partitioned into 4 bins, U,Vchromaticity channels may be partitioned into 8 bins each therebyproducing 18 bins. In one or more implementations, number of bins withina color space may be selected between 2 and 200. Histogram determinationmay comprise comparison of individual pixel values within the image toindividual bin ranges of the color space.

At operation 1204 a back-projected version of the image may bedetermined using the histogram information. In some implementations, theback-projection operation may comprise assigning a saliency value at alocation (x,y) determined based on a value of the respective bin of thehistogram the may correspond to pixel value within the original image atthe location (x,y).

At operation 1206 saliency map may be determined based on theback-projected image information. Histogram bin values may conveyinformation related to frequency of occurrence of a given pixel valuewithin the image. Lower frequency of occurrence may correspond greaterthe saliency, in accordance with some implementations. In someimplementations, saliency map determination may comprise one or more ofspatial filtering (e.g., Gaussian blur), thresholding, and/or otheroperations configured to increase difference between an area of highsaliency relative background (signal to noise) within the saliency map.

FIG. 13 illustrates a method of navigating a trajectory by a roboticdevice using salient feature detection, in accordance with one or moreimplementations. Operations of method 1300 may be performed by, e.g.,autonomous robotic vehicle 160 of FIG. 1 when approaching a target 174,and/or vehicle 1000 when landing while avoiding an obstacle in FIG. 10in one or more implementations.

At operation 1302 an image may be obtained by the robotic device whilenavigating a trajectory associated with a task. In some implementations,the image may comprise a plurality of pixels characterized by one ormore channels (e.g., the image 200, 230 in FIGS. 2A-2B).

At operation 1304 saliency map may be determined based on determining adeviation of a characteristic within an area of the image relative thecharacteristic of other area(s). In one or more implementations, thedeviation determination may comprise determination of color distancedifference, analysis of difference between a value of a pixel and valuesof pixels in a surrounding region, analysis of pixel likelihood ofoccurrence, motion magnitude and/or direction, circularity, orientationof pixel patches, and/or other operations. In some implementations, thesaliency map may be determined based on a combination (e.g.,multiplicative, and/or additive) of two or more maps, e.g., as describedabove with respect to FIGS. 5-6.

At operation 1308 location of an object may be determined based on thesaliency map evaluation. In some implementations, the evaluation maycomprise determination of color, orientation, dimensions (e.g.,circularity), surface area, motion direction and/or magnitude, and/orother parameters associated with the salient area.

At operation 1310 the trajectory may be adjusted based on the detectedobject feature. In some implementations, wherein the detected object maycomprise a target (e.g., 174 in FIG. 1) the trajectory adjustment may beconfigured to guide the robotic device closer to the object. In someimplementations, wherein the detected object may comprise an obstacle(e.g., 1010 in FIG. 10) the trajectory adjustment may be configured toguide the robotic device away from the object.

Implementations of the principles of the disclosure may be furtherapplicable to a wide assortment of applications including computer-humaninteraction (e.g., recognition of gestures, voice, posture, face, and/orother interactions), controlling processes (e.g., processes associatedwith an industrial robot, autonomous and other vehicles, and/or otherprocesses), augmented reality applications, access control (e.g.,opening a door based on a gesture, opening an access way based ondetection of an authorized person), detecting events (e.g., for visualsurveillance or people or animal counting, tracking).

A video processing system of the disclosure may be implemented in avariety of ways such as, for example, a software library, an IP coreconfigured for implementation in a programmable logic device (e.g.,FPGA), an ASIC, a remote server, comprising a computer readableapparatus storing computer executable instructions configured to performfeature detection. Myriad other applications exist that will berecognized by those of ordinary skill given the present disclosure.

Although the system(s) and/or method(s) of this disclosure have beendescribed in detail for the purpose of illustration based on what iscurrently considered to be the most practical and preferredimplementations, it is to be understood that such detail is solely forthat purpose and that the disclosure is not limited to the disclosedimplementations, but, on the contrary, is intended to covermodifications and equivalent arrangements that are within the spirit andscope of the appended claims. For example, it is to be understood thatthe present disclosure contemplates that, to the extent possible, one ormore features of any implementation may be combined with one or morefeatures of any other implementation.

What is claimed is:
 1. A robotic apparatus configured to track a targetobject comprising: a camera; and a non-transitory computer-readablemedium having computer-readable instructions stored thereon, that whenexecuted by at least one processor apparatus, configure the at least oneprocessor apparatus to, obtain a plurality of images via the camerawhile navigating a trajectory associated with a physical task; identifya salient area in the plurality of images, the identification of thesalient area being based at least on a contrast between (i) a relativemotion associated with the target object and (ii) a relative motionassociated with another object within a field of view of the camera, thetarget object being within the field of view, where the relative motionassociated with the target object is determined from a combination of amotion component associated with a movement of the object and a motioncomponent associated with the navigation of the trajectory of therobotic apparatus; initialize an object tracking process based at leaston the identification of the salient area, the object tracking processcomprising a detection of the object from the salient area; and adjustthe trajectory associated with the physical task based on the detectionof the object from the salient area.
 2. A non-transitorycomputer-readable medium having computer readable instructions storedthereon, that when executed by at least one processor, configure the atleast one processor to, determine a relative motion of an object in asequence of one or more images when a robotic apparatus is in motionalong a trajectory, the relative motion of the object being determinedbased on (i) determining a first motion component, (ii) determining asecond motion component, and (iii) combining the first motion componentand the second motion component, the first motion component beingassociated with the motion along the trajectory of the roboticapparatus, the second motion component being associated with a movementor an absence of the movement of the object, the object being locatedwithin a visual scene; identify a salient area in the sequence of one ormore images based on the determined relative motion of the object, thesalient area identified being based on determining a contrast between(i) the determined relative motion of the object and (ii) a relativemotion of at least one other object within the visual scene, therelative motion of the at least one other object being based on thefirst motion component and a third motion component, the third motioncomponent being associated with a movement or an absence of the movementof the at least one other object; and initialize an object trackingprocess based at least on the identifying of the salient area.
 3. Thenon-transitory computer readable medium of claim 2, wherein the at leastone processor is further configured to execute the computer readableinstructions to reduce a number of occurrences of false positivesassociated with detecting the object in subsequent one or more images ofthe plurality of images when the object tracking process is initialized,as compared to detecting the object in the subsequent one or more imagesin absence of the initializing.
 4. The non-transitory computer readablemedium of claim 3, wherein, the one or more images include individualpixels, the individual pixels characterized by pixel characteristics,and the at least one processor is further configured to execute thecomputer readable instructions to determine a saliency distribution ofpixels of the one or more images of the sequence of images based on atleast one of the one or more pixel characteristics during identifying ofthe salient area.
 5. The non-transitory computer readable medium ofclaim 4, wherein, the one or more pixel characteristics comprises apixel chromaticity parameter; and the at least one processor is furtherconfigured to execute the computer readable instructions to determine aspatial distribution of pixel color distance in the one or more imageswhen the saliency distribution is determined, the pixel color distancebeing determined based at least on a distance measure between thechromaticity parameter of individual pixels of the one or more imagesand a reference chromaticity.
 6. The non-transitory computer readablemedium of claim 5, wherein the reference chromaticity is determinedbased at least on determining a pixel chromaticity occurrence statisticsparameter.
 7. The non-transitory computer readable medium of claim 5,wherein the identification of the salient area in the one or more imagescomprises causing the at least one processor to execute the computerreadable instructions to, determine that one or more values of thespatial distribution of pixel color distance breach a threshold; anddetermine a thresholded pixel distance map based at least on determininga location of one or more pixels within the one or more images, thelocation corresponding to the one or more values of the spatialdistribution of pixel color distance.
 8. The non-transitory computerreadable medium of claim 7, wherein the identification of the salientarea in an image further comprises, the at least one processor isfurther configured to execute the computer readable instructions to,determine a low-pass filtered version of the thresholded pixel distancemap; and determine a shape parameter associated with at least a portionof the low-pass filtered version.
 9. The non-transitory computerreadable medium of claim 8, wherein the identification of the salientarea in the image further comprises causing the at least one processorto execute the computer readable instructions to, determine one or morecontours associated with one or more contiguous pixel groups within thelow pass filtered version; and for a given contour of the one or moreindividual contours, the at least one processor is further configured toexecute the computer readable instructions to, determine a smallestenclosing circle; and determine a circularity based at least on a ratiobetween an area of the given contour and an area of the smallestenclosing circle.
 10. The non-transitory computer readable medium ofclaim 9, wherein the at least portion of the low-pass filtered versionis selected based at least on an evaluation of a greatest contiguousnumber of pixels.
 11. The non-transitory computer readable medium ofclaim 8, wherein the at least portion of the low-pass filtered versionis selected based at least on a greatest cumulative saliency value, thevalue determined by accumulating a plurality of saliency values.
 12. Thenon-transitory computer readable medium of claim 3, wherein theidentification of the salient area in the one or more images comprisescausing the at least one processor to execute the computer readableinstructions to, partition the image into a plurality of partitions,individual ones of the plurality of partitions comprising a plurality ofadjacent pixels in a contiguous area; convolve individual ones of theplurality of partitions with one or more spatial filter masks to producea plurality of convolution magnitudes, individual ones of the one ormore spatial filters characterized by filter orientation; and determinethe salient area corresponding to a least frequently occurring filterconvolution magnitude above a threshold.
 13. The non-transitory computerreadable medium of claim 3, wherein the salient area is identified basedat least on analysis of displacement of pixels of the one or more imagesrelative to pixels of a preceding image of the sequence of images. 14.The non-transitory computer readable medium of claim 13, wherein: theanalysis of displacement comprises causing the at least one processor toexecute the computer readable instructions to, determine a motion map ofpixels of the one or more images, the motion map at least characterizingmagnitude and direction of motion of pixels in the one or more imagesassociated with representation of the object; and determine the saliencymap based at least on an additional information related to motion of theobject.
 15. The non-transitory computer readable medium of claim 14,wherein the additional information is obtained during a tracker primingoperation, the operation comprising the at least one processor isfurther configured to execute the computer readable instructions toanalyze a plurality of images characterizing motion of the object, thepriming operation comprising motion of the object in a given pattern andin absence of other sources motion in individual ones of the pluralityof images.
 16. The non-transitory computer readable medium of claim 3,wherein: the one or more images of the sequence of images are obtainedby a sensor configured to sample energy reflected from an environment;individual pixels of the one or more images are characterized by a rangechannel corresponding to a range value between the sensor and a locationwithin the environment; and the salient area is determined based atleast on information related to a given range to the object and acomparison of range channel value associated with pixels in the salientarea to the range to the object.
 17. The non-transitory computerreadable medium of claim 4, wherein: the one or more images arecharacterized by a color space comprising at least one dimension; andthe pixel characteristics are configured to convey information relatedto the at least one dimension.
 18. The non-transitory computer readablemedium of claim 17, wherein: the salient area is characterized by afirst pixel occurrence; the saliency distribution comprises a backgroundportion characterized by a second pixel occurrence, the secondoccurrence being greater than the first occurrence; and the first pixeloccurrence is configured to provide a reference within the color space.19. The non-transitory computer readable medium of claim 18, wherein:the at least one dimension comprises a chromaticity dimension; and thereference comprises a reference chromaticity.
 20. The non-transitorycomputer readable medium of claim 19, wherein the at least one processoris further configured to execute the computer readable instructions todetermine the reference chromaticity based on a distribution of pixelchromaticity occurrence within the one or more images.
 21. Thenon-transitory computer readable medium of claim 20, wherein the atleast one processor is further configured to execute the computerreadable instructions to determine a difference measure betweenchromaticity of individual pixels within the one or more images and thereference chromaticity; and the determination of the referencechromaticity based on the pixel chromaticity occurrence is configured toincrease a discriminability between a distance measure associated withpixels corresponding to the salient area of the one or more images and adistance measure corresponding to the background portion of the one ormore images.
 22. The non-transitory computer readable medium of claim21, wherein the increased discriminability is configured based on agreater difference between (i) the distance measure associated withpixels corresponding to the salient area of the one or more images, and(ii) a distance measure corresponding to the background portion of theone or more images when compared to respective distance measuresdetermined based on another reference chromaticity, the anotherreference chromacity determined based at least on pixels disposedoutside the salient area of the one or more images.